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ABSTRACT 



An imaging system is provided for imaging a scene to 
produce a sequence of image frames of the scene at a frame 
rate, R, of at least about 25 image frames per second. The 
system includes an optical input port, a charge-coupled 
imaging device, an analog signal processor, and an analog- 
to-digital processor (A/D). The A/D digitizes the amplified 
pixel signal to produce a digital image signal formatted as a 
sequence of image frames each of a plurality of digital pixel 
values and having a dynamic range of digital pixel values 
represented by a number of digital bits, B, where B is greater 
than 8. A digital image processor is provided for processing 
digital pixel values in the sequence of image frames to 
produce an output image frame sequence at the frame rate, 
R, representative of the imaged scene, with a latency of no 
more than about 1/R and a dynamic range of image frame 
pixel values represented by a number of digital bits, D, 
where D is less than B. The output image frame sequence is 
characterized by noise-limited resolution of at least a mini- 
mum number, N^, of line pairs per millimeter, referred to 
the charge-coupled imaging device pixel array, in an imaged 
scene as a function of illuminance of the input light imping- 
ing the charge-coupled imaging device pixels. 

22 Claims, 22 Drawing Sheets 
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REAL TIME ADAPTIVE DIGITAL IMAGE 
PROCESSING FOR DYNAMIC RANGE 
REMAPPING OF IMAGERY INCLUDING 
LOW-LIGHT-LEVEL VISIBLE IMAGERY 

This application is a continuation of Ser. No. 08/632,746 
Apr. 15, 1996. 

GOVERNMENT RIGHTS IN THE INVENTION 

This invention was made with support under Contract 
Number F19628-95-C-0002 awarded by the Air Force. The 
U.S. Government has certain rights in this invention. 

FIELD OF THE INVENTION 

This invention relates to digital imaging and image pro- 
cessing techniques, and more particularly relates to digital 
imaging of low- light- level scenes at real time video image 
display speeds. 

BACKGROUND OF THE INVENTION 

Imaging of low-light-level environments is an important 
capability for enabling military and law enforcement 
surveillance, aviation and automotive navigation, and for 
numerous industrial and consumer manufacturing and pro- 
duction processes, among other applications. For example, 
the conventional low-light-level imaging systems known as 
so-called night vision scopes are routinely employed as a 
principal means for enabling night time mobility and navi- 
gation by military personnel wearing helmet-mounted 
scopes while traveling on foot or navigating in a vehicle, 
e.g., a jeep, truck, helicopter or jet. 

Historically, low-light- level imaging systems, including 
conventional night vision scopes, have been based on use of 
an electro-optic image sensor that provides a gain mecha- 
nism for amplifying ambient input light to produce an output 
image signal level that is adequate for visual display, e.g., 
either by a direct view display or, with the addition of an 
electronic imaging system, a remote view display. For 
example, in a typical electro-optic system such as a so-called 
intensifier tube imager, a lens is used to focus ambient light, 
such as moonlight or starlight reflected off of a scene, onto 
a photocathode in a vacuum tube, the photocathode being 
sensitive to, e.g., light from the yellow through near-infrared 
portion of the electromagnetic spectrum. Under application 
of a high voltage between the photocathode and a micro- 
channel plate also located in the vacuum tube, an input 
ambient photon incident on the photocathode causes emis- 
sion of a single electron from the photocathode and accel- 
eration of the electron toward the micro-channel plate. Upon 
striking the micro-channel plate, the single electron creates 
a cascade of many electrons, which together are accelerated 
toward a phosphor screen by a second applied voltage. The 
kinetic energy of the electrons striking the phosphor causes 
phosphor to glow. This electron cascade mechanism results 
in amplification of the input ambient light, typically by about 
four orders of magnitude, to produce a visible image on the 
phosphor screen. The phosphor image exists only momen- 
tarily in the glow of the phosphor screen, and does not exist 
in a storable or readable form. 

To produce a real-time electronic video image, which 
typically is set at a frame rate of about 30 frames/second, 
based on the optical images formed by an intensifier tube on 
a phosphor screen, it is common practice to optically couple 
the phosphor screen to a conventional charge-coupled- 
device (CCD) electronic imaging camera. Optical coupling 
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is typically achieved using an intermediate coupling lens or 
an optical fiber taper bonded to the CCD and either bonded 
or integrally-connected to the intensifier tube phosphor 
screen. The resulting low-light video camera, or so-called 

5 intensified-CCD camera, relies entirely on the cascade gain 
mechanism of the intensifier tube to provide a phosphor 
image that is adequately amplified to be sensed by the 
conventional CCD imager. 

Intensified-CCD cameras like the one described above, 

10 while capable of producing a real time low-light -level video 
sequence, have historically been severely restricted with 
regard to other performance criteria. In particular, the intra- 
scene dynamic range of an image produced by an 
intensified-CCD imaging system is severely limited by the 

15 image intensifier tube. Furthermore, image resolution is 
severely degraded at low light levels due to electronic noise 
associated with the intensifier tube cascading gain mecha- 
nism. This electronic noise adds background image intensity 
noise and can even "swamp" low intensity images, resulting 

20 in an output image that is a poor rendition of the imaged 
scene. Intensifier tube imagers not including a CCD elec- 
tronic camera are of course also subject to the resolution and 
intra-scene dynamic range limitations imposed by the inten- 
sifier tube gain mechanism. 

25 These limitations are exacerbated in imaging low-light- 
level scenes because the same scene may contain very low 
brightness areas as well as dramatic intra-scene intensity 
fluctuations due, e.g., to man-made light. But because inten- 
sifier tube imagers and intensified-CCD imaging systems 

30 mtrinsically rely on the vacuum tube cascading gain mecha- 
nism for production of a viable phosphor image, the 
dynamic range limitation imposed by the gain mechanism 
must be accepted, resulting in either loss of darker areas in 
the scene or excessive blooming in the brighter areas of the 

35 scene. Blooming is here meant as a localized brightness 
saturation that spills over to other nearby areas. As a result, 
intensifier tube imagers and intensified-CCD imaging sys- 
tems are restricted to relatively small dynamic range imag- 
ing; typically no more than about 200 gray levels can be 

40 enabled by even the best vacuum tube-based systems, and 
generally, far fewer gray levels span the restricted intra- 
scene dynamic range. 

Additional inherent limitations of intensifier vacuum tube 
technology limit the overall performance of intensifier tube 

45 imagers and intensified-CCD imaging systems. For 
example, the finite time required for a phosphor image 
produced by an intensifier tube to dissipate from the phos- 
phor results in deleterious image artifacts in a temporal 
sequence of images when there is motion in the scene. In 

50 addition, vacuum tubes, being formed of glass, are fragile, 
and therefore require special handling considerations for the 
image system in which they are incorporated. The photo- 
cathode and micro-channel plate used in the intensifier tube 
have relatively short life cycles, requiring frequent replace- 

55 ment and repair. Furthermore, vacuum tubes are relatively 
large in size, limiting the minimum overall imaging system 
size. Vacuum tubes are also relatively expensive, adding 
significant cost to the overall cost of the imaging system. 
Many other performance, handling, and packaging limita- 

60 tions are additionally imposed by the intensifier vacuum 
tube technology. 

Another class of low-light -level imaging systems, known 
as so-called slow-scan or frame-integrating cameras, do not 
rely on a gain mechanism to amplify ambient light for 

65 producing a viable electronic image. Instead, a slow-scan 
camera typically includes only a CCD imager that in opera- 
tion is exposed to a low-light-level scene for an extended 
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period of lime, i.e., seconds or longer, during which time the 
device accumulates a large number of photoelectrons; after 
a time period sufficient to accumulate an adequate photo- 
electron count, a viable electronic image can be produced by 
the CCD camera. This technique overcomes some of the 5 
practical limitations of intensifier tube imagers and 
intensified-CCD image systems, but is inherently limited to 
extremely slow image capture speeds. Slow-scan cameras 
thus cannot accommodate real time imaging for production 
of video sequences at rates even close to about 25-30 frames 10 
per second. Indeed, slow-scan cameras are typically used in 
applications that are not primarily time-sensitive; for 
example, being used in astronomical applications as an 
electronic substitute for long-exposure film photography 
through telescopes. 15 

Many critical low-light-level surveillance and mobility 
applications require real time digital video imaging of night 
time scenes, e.g., air-land scenes, characterized by a large 
intra-scene dynamic range. But like the systems described 
above, conventional low-light-level imaging systems devel- 20 
oped heretofore have provided only suboptimal performance 
under such complex conditions and are further restricted by 
additional performance and practical limitations that impede 
or inhibit a high level of operational performance in appli- 
cations for which real time low-light-level imaging is criti- 25 
cal. 

SUMMARY OF THE INVENTION 

The invention overcomes limitations of past low-light- 
level imagers to provide an imaging system that attains 30 
superior performance at real time speeds. In a first aspect, 
the invention provides an imaging system for imaging a 
scene to produce a sequence of image frames of the scene at 
a frame rate, R. The imaging system includes an optical 
input port for accepting input light from the scene and a 35 
charge-coupled imaging device including pixels configured 
in a charge storage medium, the charge-coupled imaging 
device located in relation to the input port such that input 
light from the scene impinges device pixels. The charge - 
coupled imaging device produces an electrical pixel signal 40 
of analog pixel values based on the input light. In the 
invention, an analog signal processor is connected to the 
charge-coupled imaging device for amplifying the pixel 
signal, and an analog-to-digital processor is connected to the 
analog signal processor for digitizing the amplified pixel 45 
signal to produce a digital image signal formatted as a 
sequence of image frames each of a plurality of digital pixel 
values and having a dynamic range of digital pixel values 
represented by a number of digital bits, B. Finally, the 
imaging system includes a digital image processor con- 50 
nected to the analog-to-digital processor for processing 
digital pixel values in the sequence of image frames to 
remap the dynamic range of the frames to a compressed 
dynamic range of remapped pixel values represented by a 
number of digital bits, D, where D is less than the number, 55 
B. A sequence of output image frames of remapped pixel 
values representative of the imaged scene are thereby pro- 
duced at the frame rate, R, with a latency time of no more 
than about 1/R. 

In preferred embodiments, the imaging system includes a 60 
display connected to receive the output image frame 
sequence and to display the sequence at the frame rate, R. 
Preferably, the digital image processor of the imaging sys- 
tem consists of a center-surround-shunt processor for adap- 
tively enhancing contrast of digital pixel values from the 65 
analog-to-digital processor based on values of neighboring 
pixels in an image frame, and for adaptively normalizing the 
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enhanced pixel values such that the enhanced pixel values 
are within a compressed and normal ized dynamic ran ge. The | 
digital image processor further preferably includes asTatis- 
tics processor for acquiring pixel value statistics about the 
digital pixel values from the analog-to-digital processor and 
for acquiring pixel value statistics about the enhanced and 
normalized pixel values from the center-surround-shunt 
processor, and a remapping function processor for construct- 
ing a pixel value remapping function based on the pixel 
value statistics acquired by the statistics processor, the 
remapping function constituting a rule for remapping the 
enhanced and normalized pixel values from the center- 
surround-shunt processor to a selected output dynamic range 
represented by the number of digital bits D. The digital 
image processor further preferably includes a remap pro- 
cessor for applying the remapping function from the remap- 
ping function processor to the enhanced and normalized 
pixel values from the center-surround-shunt processor to 
produce a sequence of output image frames of remapped 
pixel values representative of the imaged scene. Preferably, 
the remapping function processor constructs a pixel value 
remapping function for a given image frame before pixel 
values in that frame are processed by the center-surround- 
shunt processor. 

In other preferred embodiments, the number of digital 
bits, B, representing the dynamic range of digital pixel 
values produced by the analog-to-digital processor is greater 
than 8 and the number of digital bits, D, representing the 
compressed dynamic range of remapped pixel values is no 
larger than 8; preferably the frame rate, R, is at least about 
25 frames per second. 

In another aspect, the invention provides an imaging 
system for imaging a scene to produce a sequence of image 
frames of the scene at a frame rate, R, of at least about 25 
image frames per second. The imaging system includes an 
optical input port for accepting input light from the scene 
and a charge -coupled imaging device having an array of 
pixels configured in a charge storage medium. The charge- 
coupled imaging device is located in relation to the input 
port such that input light from the scene impinges device 
pixels, such that the charge-coupled imaging device pro- 
duces an electrical pixel signal of analog pixel values based 
on the input light. An analog signal processor is connected 
to the charge-coupled imaging device for amplifying the 
pixel signal, and an analog-to-digital processor is connected 
to the analog signal processor for digitizing the amplified 
pixel signal to produce a digital image signal formatted as a 
sequence of image frames each of a plurality of digital pixel 
values and having a dynamic range of digital pixel values 
represented by a number of digital bits, B, where B is greater 
than 8. A digital image processor is connected to the 
analog-to-digital processor for processing digital pixel val- 
ues in the sequence of image frames to produce an output 
image frame sequence at the frame rate, R, representative of 
the imaged scene, with a latency of no more than about 1/R. 

In the invention, the output image frame sequence is 
characterized by noise- limited resolution of at least a mini- 
mum number, N^, of line pairs per millimeter, referred to 
the charge-coupled imaging device pixel array, in an imaged 
scene as a function of illuminance of the input light imping- 
ing the charge-coupled imaging device pixels. For a scene 
characterized by a contrast of about 0.3, for a human 
observation time of about 0.05 seconds, and for an image 
scene frame rate of about 30 frames per second, N^ is given 
as N^l.900 L 0 ' 51 , where L is the value of illuminance of the 
input light impinging the charge-coupled imaging device 
pixels, for at least one illuminance value between a range of 
illuminance values of about lxlO -2 LUX and 5xl0" 7 LUX. 
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la preferred embodiments of the invention, L, the illumi- 
nance of the input light impinging the charge-coupled imag- 
ing device pixels, ranges between about lxlCT 3 LUX and 
lxlO -6 LUX. Preferably, the frame rate, R, is at least about 
30 frames per second. In preferred embodiments, the optical 5 
input port comprises a lens, and a display is connected to 
receive the output image frame sequence and to display the 
sequence at the frame rate, R. In other embodiments, the 
digital image processor and the display are in communica- 
tion with but located remote from the charge-coupled imag- 10 
ing device and the analog-to-digital processor; alternatively, 
the charge-coupled imaging device, the analog-to-digital 
processor, and the display are in communication with but 
located remote from the digital image processor and a power 
supply. In other embodiments, a communication link is 15 
connected to the digital image processor for transmitting to 
a remotely located receiver the sequence of output image 
frames; a user controller is preferably included for control- 
ling the frame rate, R, and resolution of the imaging system, 
within operational limits of the imaging system. 20 

In other embodiments, the charge -coup led imaging device 
pixels are configured in a charge storage substrate having a 
front side supporting pixel interconnections and a back side 
having no substantial topology, with buried channels in the 
substrate defining charge packet storage wells for the pixels. 25 
Preferably the charge-coupled imaging device is located in 
relation to the optical input port such that input light from 
the scene impinges the back side of the substrate. In other 
preferred embodiments, the substrate is a silicon substrate 
characterized by a resistivity of at least about 1000 Q-cm. 30 

Preferably, the electrical pixel signal of pixel values has 
a dynamic range of at least about 1000 distinct pixel value 
levels and most preferably a dynamic range of at least about 
3000 distinct pixel value levels. 

... 35 

In other embodiments, the configuration of pixels in the 
charge storage substrate is configured as an imaging pixel 
array on which input light impinges to produce charge 
packets in the buried channels of the imaging array pixels 
and a frame storage pixel array shielded from impinging ^ 
input light, charge packets in the imaging pixel array being 
transferred to the frame storage pixel array for producing an 
electrical pixel signal of analog pixel values based on the 
input light. Preferably, the imaging pixel array consists of an 
array of electronically-shuttered pixels, each pixel in the 45 
array being selectively electronically controllable by the 
pixel interconnections to inhibit storage of charge packets in 
the buried channel of that pixel while input light impinges 
the charge-coupled imaging device. 

In other preferred embodiments, the pixel interconnec- 50 
tions supported on the front side of the charge storage 
medium define a three-phase clocking configuration for 
transferring charge packets in the buried channels; and the 
pixel interconnections supported on the front side of the 
charge storage substrate provide interconnections for selec- 55 
tive electronic transfer of a charge packet in a given pixel of 
the imaging pixel array to an adjacent pixel located in a 
different row of the imaging pixel array, and provide inter- 
connections for selective electronic transfer of a charge 
packet in a given pixel in the imaging array to an adjacent 60 
pixel located in a different column of the imaging pixel 
array. In other embodiments, charge packets resident in a 
portion of the pixels in the frame storage pixel array are 
summed prior to production of an electrical pixel signal of 
analog pixel values. 65 

In other preferred embodiments, the pixels of the charge - 
coupled imaging device are configured in a charge storage 
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medium comprising a substrate that is curved in a selected 
nonplanar focal surface profile and located a selected dis- 
tance from the lens with the focal surface facing the lens, the 
focal surface profile and lens-to-substrate distance selected 
such that the input light is in focus at the location of the 
substrate. Preferably, a cooling device is provided in contact 
with the charge storage substrate to suppress dark current 
charge packet generation in pixels of the charge-coupled 
imaging device. The cooling device preferably consists of a 
thermo-electric cooling device. 

In another aspect, the invention provides an imaging 
system having optical input port for accepting input light 
from the scene, a first charge-coupled imaging device having 
pixels configured in a charge storage medium and located in 
relation to the input port such that at least a central field- 
of-view region of the input light impinges device pixels to 
produce an electrical central field-of-view pixel signal of 
analog pixel values based on the input light, and a second 
charge -coupled imaging device having pixels configured in 
a charge storage medium and located in relation to the input 
port such that at least a peripheral field-of-view region of the 
input light impinges device pixels to produce an electrical 
peripheral field-of-view pixel signal of analog pixel values 
based on the input light. An image processor is connected to 
receive the central field-of-view pixel values and peripheral 
field-of-view pixel values to amplify and digitize the pixel 
values and to blend the central field-of-view pixel values 
with the peripheral field-of-view pixel values to produce a 
sequence of composite image frames, each composite image 
frame having digital central field-of-view pixel values in a 
central region of the composite image frame and having 
digital peripheral field-of-view pixel values surrounding the 
central region to form a peripheral region of the composite 
image frame. The sequence of composite image frames is 
produced at the frame rate, R, with a latency time of no more 
than about 1/R. 

In preferred embodiments, a display is connected to 
receive the sequence of composite image frames and display 
the sequence at the frame rate, R, with the central image 
region of each composite image frame displayed at unity 
magnification and the peripheral image region of each 
composite image frame displayed at a magnification less 
than unity. Preferably, the display subtends a field of view, 
with respect to a display viewer, that exceeds the field of 
view subtended by the central image region, and preferably, 
the central image region subtends an angle of at least about 
30 degrees and the peripheral image region subtends an 
angle of at least about 80 degrees. 

In preferred embodiments, the system includes a field- 
of-view separator aligned with the optical input port for 
directing the central field-of-view region of the input light to 
the first charge coupled imaging device and for directing the 
peripheral field-of-view region of the input light to the 
second charge coupled imaging device; the system also 
preferably includes a long focal length lens located between 
the field-of-view separator and the first charge-coupled 
imaging device to focus the central field-of-view region of 
the input light onto the first charge -coupled imaging device; 
and a short focal length lens located between the field-of- 
view separator and the second charge -coupled imaging 
device to focus the peripheral field-of-view region of the 
input light onto the second charge-coupled imaging device. 

The invention provides, in another aspect, a charge- 
coupled imaging device for imaging a wide field-of-view 
scene to produce a sequence of image frames of the scene at 
an image frame rate, R, the imaging device consists of a 
short focal length lens for accepting light from the scene to 
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be imaged and a charge storage medium consisting of a The reset charge funnel is defined by a buried implant 

charge storage substrate having a front side and a back side, having a first width at the end of the reset buried channel and 

the charge storage substrate being curved in a selected a second width at the charge-collection junction, the first 

nonplanar focal surface profile and located a selected dis- width being larger than the second width. Preferably, the 

tance from the lens with the focal surface facing the lens, the 5 charge-collection junction consists of a dopant region in the 

focal surface profile and lens-to-substrate distance selected substrate, dopant in the region provided by diffusion of 

such that the light accepted by the lens is in focus at the dopant from a doped conducting layer deposited over the 

position of the substrate, A support substrate is provided on charge-collection junction location into the substrate under 

which the nonplanar charge storage substrate is supported to me conducting layer at the charge-collection junction loca- 

maintain the selected surface profile of the charge storage 10 ^ on 
substrate. An array of pixels is defined in the charge storage 

substrate by pixel interconnections supported on the front BRIEF DESCRIPTION OF THE DRAWINGS 
side of the substrate, exposure of the substrate to light from 

the scene through the lens producing charge packets in the FIG- 1 shows a ° operational block diagram of an imaging 
pixels, the pixel interconnections providing selective elec- 15 svstem m accordance with the invention; 
tronic temporal control of transfer of charge packets from FIG. 2A shows a diagrammatic representation of a hand- 
one pixel to another in the substrate. Means for suppressing held embodiment of the imaging system of the invention; 
generation of dark current charge packet generation in the FIG. 2B shows an operational block diagram of the 
substrate pixels is provided, as well as an output circuit for embodiment of the imaging system of FIG. 2 A; 
converting the charge packets in the pixels to an electrical 20 FIGS 2 C-2D show diagrammatic representations of bio- 
pixel signal of output pixel values based on the light from cular and binocu i ar embodiments of the imaging system of 
the scene, a plurality of pixel values together forming an ^ invention> respectively; 

image frame, the output pixel values bemg produced at a r^ 2£ diagrammauc representation of a first 

corresponding to the image frame rate, R. vehicle-mounted embodiment of the imaging system of the 

The invention further provides a charge -coupled imaging 2 s invention- 
device consisting of a charge storage medium comprising a ' 

substrate having a front side and a backside, pixels defined ™ G <. 2F sh f ows aD operational block diagram of the 

in an array of pixel rows and pixel columns in the charge embodiment of the imaging system of FIG. 2E; 

storage substrate by a plurality of columns of buried chan- FIG. 2G shows a diagrammatic representation of a second 

nels in the substrate and by a plurality of rows of pixel gates 30 vehicle-mounted embodiment of the imaging system of the 

on the substrate over the buried channel columns. A channel invention; 

stop region is provided in the substrate between and at the FIG. 2H shows a diagrammatic representation of a multi- 
periphery of the columns of buried channels, and a serial site embodiment of the imaging system of the invention; 
output register is defined by a row of register gates on the p IG 3a shows a diagrammatic representation of a CCD 
substrate over the buried channels and adjacent to the last 35 imager in accordance with the invention; 
pixel row in the pixel array. The output register includes an nG 3B shows a view of a three - P hase 
output stage defined by an output stage gate at the end of the CCD ^ ^ accordance ^ the mvention; 
register gate row and a corresponding output stage buried , . , . - , 
channel comprising an end column of the plurality of buried FIG . 3C shows a cross-sectional view of a CCD charge 
channel columns, the output stage buried channel extending 40 folkcbon node and its vicinity in accordance with the 
to the output stage gate. A charge-collection junction is invention, 

provided adjacent to the output stage gate and defined by a FIGS. 4A-4C show a top plan view, a cross-sectional 

p/n junction in the substrate for collecting charge generated v * ew alon g une A " A '> and a potential well diagram corre- 

in the array of pixels and output at the output stage gate. sponding to the cross-sectional view, respectively, for a 

An output stage charge funnel is located between the 45 blooming drain configuration in accordance with the inven- 

output stage gate and the charge -collection junction for uon » 

tunneling charge in the output stage buried channel to the FIG. 5 is a circuit diagram for a CCD imager output 

charge-collection junction. The output stage charge funnel is circuit in accordance with the invention; 

defined by a buried implant having a first width at the end FIGS. 6A and 6B show a top plan view and cross- 

of the buried channel and a second width at the charge- 50 sectional view along line A-A, respectively, for an elec- 

collection junction, the first width being larger than the tronic pixel shutter in accordance with the invention; 

second width. pj G 7 ^ a cross-sectional view of a CCD imager output 

Preferably, an output circuit is connected to the charge- circuit FET in accordance with the invention; 

collection junction for converting the collected charge to an F]G g fc a cro& ^ ctiom] view mustrating a parasitic 

electronic representation of a scene bemg imaged, the output 55 capacitance due t0 C CD pixel channel stop regions; 

circuit including a capacitor, an output transistor, and a reset „ T „ n . . . _ „„„ , . , , . , 

n 1 . K . 1 u * ♦ tu v • FIG. 9 is a top plan view of a CCD buried channel charge 

transistor all located in the substrate. The capacitor is . , j * * • ■* rr-r t. r. 1 j ^ - 

L , 4 , M c j funnel and output circuit FET charge funnel provided in 

connected to the charge-collection junction tor producing a , . 5 , • ^ . 

a - r 1 11 * j accordance wnn me invention, 
voltage corresponding to a given amount of charge collected 

at the charge-collection junction. The reset transistor is 60 FIGS IGAr-lOD show cross-sectional views of process 

defined by a reset gate and a reset buried channel in the ^P 5 for producing a back-illuminated CCD imager m 

substrate and extends between a reset transistor bias contact accordance with the invention; 

and the reset gate. A charge funnel is located between the FIG. 11 shows a diagrammatic perspective view of a CCD 

reset gate and the charge-collection junction for draining imager in accordance with the invention; 

charge from the output stage capacitor after the output 65 FIG. 12A shows a diagrammatic representation of a 

circuit has produced an electronic representation for a given multi-element wide -field -of -view lens system employed 

amount of charge collected at the charge-collection junction. with a planar CCD imager in accordance with the invention; 
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FIG. 12B shows a diagrammatic representation of a 
wide-field-of-view single-lens employed with a nonplanar 
CCD imager in accordance with the invention; 

FIG. 12C shows a diagrammatic cross-section of a non- 
planar CCD imager substrate mounted on a support substrate 
in accordance with the invention; 

FIG. 12D shows a diagrammatic cross-sectional view of 
a configuration for producing the nonplanar CCD imager 
substrate of FIG. 12C; 

FIG. 13 shows a diagrammatic top plan view of a CCD 
imager pixel having orthogonal transfer gate interconnec- 
tions provided in accordance with the invention; 

FIG. 14 shows a cross-sectional view of a CCD imager 
packaging and cooling configuration in accordance with the 
invention; 

FIG. 15 is a circuit diagram of a signal processing circuit 
provided in accordance with the invention; 

FIG. 16A shows timing diagrams for controlling the CCD 
imager of the invention; 

FIG. 16B shows a state machine diagram for controlling 
the CCD imager of the invention; 

FIGS. 16C-16D show timing diagrams for controlling the 
operation of the imaging system of the invention; 

FIG. 17 is an operational block diagram of a digital image 
processor in accordance with the invention; 

FIGS. 18A-18B are example matrices to be employed by 
the digital image processor of FIG. 17 for compressing the 
dynamic range of a digital image; 

FIG. 19A shows three starlit scenes each having 8 bits of 
dynamic range selected from original 12-bit imagery; 

FIG. 19B shows the original 12-bit imagery that produced 
the 8-bit scenes of FIG. 19A, in this case after being 
processed by the digital image processor of FIG. 17 to 
produce adaptively processed 8-bit imagery; 

FIG. 20 shows a plot of quantum efficiency of the imaging 
system of the invention and a conventional low-light-level 
imager, as a function of wavelength; 

FIGS. 21 A and 21B show photographs of starlit images 
produced by the imaging system of the invention and a 
conventional low-light-level imager, respectively. 

FIG. 22 shows a plot of calculated limited resolution in 
line-pairs/mm as a function of imager illuminance in LUX 
for the imaging system of the invention and a conventional 
low-light-level imager; and 

FIG. 23 shows a diagrammatic illustration of a multi 
field-of-view imaging system in accordance with the inven- 
tion. 

DETAILED DESCRIPTION 

Alow-Hght-level imager in accordance with the invention 
provides high performance imaging of low-light-level 
scenes such that a high degree of image data can be 
perceived by human viewing of the scenes, and does so 
without the limitations imposed by prior low-light-level 
imagers. Referring to FIG. 1, this is accomplished in the 
invention by an imaging system 10 that inherently does not 
require a gain mechanism to adequately image a low-light- 
level scene and that can image such a scene at speeds 
commensurate with real time digital video rates. Real time 
digital video rates are here meant to correspond to video 
frame rates of greater than about 25 frames/second, and 
preferably to correspond to rates at least as high as the U.S. 
standard television frame rate, which is about 30 frames/ 
second. The low-light-levels accommodated by the imaging 
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system of the invention can correspond to low brightness, 
low contrast, scenes such as night time scenes illuminated 
primarily by moonlight, starlight, or even overcast starlight, 
or day time scenes in foggy, smoky, hazy, or other such low 

5 contrast environments. 

As shown in FIG. 1, the imaging system 10 accepts 
ambient input light 12 from a scene. The input light 12 is 
accepted at an optical input port 14 that admits the light and 
directs the light to one or more imagers 16a, 16b, . . . 16/i. 

10 As explained below, the optical input can consist of a lens, 
transparent window, beam splitter, mirror, or other optical 
element. 

Each imager 16 consists of a solid-state charge-coupled- 
device (CCD) that provides high optical sensitivity, low 

15 image noise generation, and quite large intra-scene dynamic 
range capabilities, among a host of other important features, 
that together enable the imager to sense the input light 12 
without first amplifying the input light and to image the 
input light at real time frame speeds. Details of the imager 

20 and its operation will be described later in the discussion. 
The imager 16 is powered by a power source 18, which can 
consist of a battery or other portable power supply or a fixed 
power supply. The power source 18 also provides power to 
a temperature controller 20 connected to the imager 16 for 

25 controlling the temperature of the imager. Power can be 
supplied to the remaining components of the system from 
those components having direct connection to the power 
supply, or can be directly connected to the power supply. 

3Q The power source 18 additionally provides power to a 
timing controller 22, which controls the initiation and syn- 
chronization of the various functions of the imaging system. 
The timing controller sets the speed of the input light 
acquisition operations conducted by the imager 16, as well 

35 as the other imaging functions. 

Once input light is acquired by the imager 16, one or more 
analog electronic signals corresponding to the input light are 
produced by an analog signal processor 24 connected to the 
output of the imager. The one or more analog electronic 

40 signals are then converted to corresponding digital elec- 
tronic signals by one or more analog-to-digital converters 
(A/D) and image formaters 26, which also multiplex and/or 
format the signal or signals, which represent pixel values, 
into a two dimensional image frame of pixel values. A digital 

45 image processor 28 accepts the digital electronic signal and 
processes the signal to enhance and further format the signal 
for electronically displaying an image, e.g., a two- 
dimensional image, of the acquired input light on a display 
33 connected to the digital image processor by way of 

50 suitable display drivers 31. The timing controller 22 coor- 
dinates the sequential processing of the acquired input light 
through the processors based on input to the timing control- 
ler from the digital image processor. 

Optionally, a user controller 32 is provided to enable a 

55 user to set various operational parameters of the imaging 
system. For example, the user controller can provide user 
control of the imager frame rate and spatial resolution, the 
brightness and contrast of the display, processing parameters 
of the digital image processor, as well as other processing 

60 parameters. Finally, a communications link 35 is provided 
for, e.g., sending acquired images as image data 36 to a 
remote location, as well as for receiving image data to be 
locally displayed. Each of the interconnected imaging sys- 
tem components will be described in turn in detail below. 

65 The imaging system of the invention can be embodied in 
a wide range of component and portable housing configu- 
rations that enable many local- and remote-access low-light- 
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level imaging scenarios, thereby accommodating a wide remotely located from other of the system components; this 

number of custom requirements for various specific imaging affords great flexibility in configuring the system for various 

applications. In particular, the components of the imaging applications. In general, only the optical input 14 of the 

system can be segmented into local and remote components, system, such as a lens, and the system imager 16 need be in 

depending on the intended imaging application. 5 a fixed spatial relationship relative to each other, that rela- 

Referring to FIG. 2A, in a first example configuration of tionship being one which accommodates acceptance of 

the imaging system of the invention, the system is embodied ambient light through the optical input 14 and focusing of it 

as a hand-held imaging scope 500. Here the operator holds onto the imager 16. The remaining components can be 

the scope to image a scene 510 that may not be visible to the housed in any convenient fashion. Of the other components, 

naked eye due to low-light-level conditions. Referring also 10 only the one or more analog signal processors 24 and one or 

to FIG. 2B, the scope 500 includes all electronics and an more A/D converters need be relatively close to the imager 

imager 16, as well as the display 33 and an output lens 516 16 to accept the electronic image data from the imager. All 

for allowing the operator to focus his gaze 518 on the other components can be located and housed remotely from 

display. The scope unit is connected by way of, e.g., a wire me imager and remotely from each other, with the restriction 

520 to the communications link unit 35 for sending and ^ mat the me thod and type of interconnection between the 

receiving image data. components maintain sufficient signal integrity between the 

In two similar configurations, shown in the top-down components to preserve operability of the overall system, 

views ^of FIGS 2C and 2D, the imaging system can be Wires> tical fibers and accompanying electro-optic 

housed as a head or helmet-mounted scope or goggle For C0DV c rt ers, transceiver systems supporting radio signal 

example the system can be configured with one ™ager(16 fl transmission, or other signal transmission and reception 

in FIG. 1) as a monocular system, or as m FIG. 2C, a 20 . t ' , °^ . , , . . 4 M . 

.. , ' M - . . ir , . 1* 1 interconnection modes can be employed to interconnect 

biocular system 525 having one imager 16 and two displays c it _ . r / 

33a, 336, or finally, as in FIG. 2D, configured with two remote components of the imager system, 

imagers 16a, 16* and two displays 33a, 33b, as a binocular Considering examples of other remote image system 

system 530. In these two examples, the minimum required configurations in accordance with the invention, in one 

electronics 540 are included in the scope housings to mini- 25 scenario, an unmanned, remotely-controlled vehicle is out- 

mize weight; other electronics, power, and other compo- fitted with a housing that supports the optical input, imager, 

nents can in these examples be housed in, e.g., a belt pack, analog signal processor, and A/D converter in forward 

body pack, or remotely, as discussed below. position or external to the vehicle, with the power source, 

The imaging system can also be configured and housed controllers, and other processors located internal to the 

for mounting on objects. For example, the system can be 30 vehicle. One or more displays are remotely connected to the 

mounted for use as a sight on a weapon or other object for internal-vehicle processors by, e.g., radio transmission. This 

which viewing of a scene is employed to enable functioning configuration enables real time navigation of an unmanned 

of the object. Referring to FIG. 2E, the imaging system can vehicle from a remote location as the vehicle traverses an 

be configured as a vehicle-mounted system 550, wherein the environment that is imaged by the imaging system to 

system is mounted internal to the vehicle 552 behind a 35 produce a view of the navigation scene to a human operator 

transparent window that allows ambient external light to be at a distant location. In a configuration similar to that shown 

acquired by the imaging system located, e.g., behind a in FIG. 2E, all system components except the display are 

vehicle windscreen such as the front window. As shown in located inside a manned or unmanned vehicle behind a 

FIG. 2F, the system here is configured with the optical input transparent window, and the displayable image produced by 

12, imager 16, and electronics 560 in a single housing, with 40 the system is transmitted to one or more remote displays that 

the display 33 and communications link 35 possibly located can each be fixed or themselves roving. In this configuration, 

at other points in the vehicle. For example, the display 33 the digital image processor can be employed to provide 

can be configured as a monitor mounted, e.g., in the fashion adaptive local control of the imaging system without the 

of a conventional rear view mirror or other configuration, or need for local human control. 

as a monitor that projects the displayed image onto the front 45 In another configuration, the optical input, imager, analog 

window, or as a heads-up display or a head-mounted display. signal processor and A/D components of the imaging system 

The communication link can be employed here for trans- are together located in a manned or unmanned land, air, 

mining an acquired image to, e.g., one or more other water, or space vehicle, with the remaining components 

vehicles or stationary command and control centers for located remotely at a fixed ground station position. Digital 

display on those vehicles' and centers' display hardware. 50 image data acquired by the vehicular imager is sent to the 

Examples of other vehicles contemplated by the invention processing components at the ground station, where a dis- 

include all types of ground vehicles, maritime vessels or playable image is produced. A display may be located at the 

submersible underwater vessels, air or space vehicles, or ground station, but one or more displays can also be located 

other vehicular means. remote from the ground station and connected to the pro- 

In an alternative vehicle mounting scheme, as shown in 55 cessing equipment in a suitable manner. With this 

FIG. 2G, the system can be configured as a gimbaled system configuration, a roving vehicle can facilitate imaging with a 

565 mounted in, e.g., a pod 568 or turret external to a minimum of on-board weight and space dedicated to the 

vehicle, with the display and communications link located at imaging components, while the remote processing station 

other points, e.g., inside the vehicle. In this configuration, provides processing and distribution of the corresponding 

the display consists of, e.g., a head-mounted display and the eo image to one or more display locations, 

imager is mounted on a motorized gimbal apparatus which An example of this scenario is shown in FIG. 2H. Here, 

in turn is slaved to the users' head orientation as a function an unmanned air vehicle acquires an image of a scene 510 

of movement of the head-mounted display. The display can using the optical input 14, imager 16, analog signal proces- 

also be embodied as a helmet-mounted or other suitable sor 24, and A/D converter 26. The resulting digital data 570 

display configuration. 65 is transmitted to a ground vehicle 572 and command station 

As illustrated by the examples just described, various 575, for digital image processing and display; each location 

components of the imaging system can be partitioned and might, in one scenario, perform different digital processing 
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to obtain different kinds of image data. The two digitally 
processed signals 578, 580 might then be transmitted to a 
remote operator 582, wearing a head-mounted display 584, 
or employing another suitable display mechanism to facili- 
tate his actions. 5 

As is apparent from this discussion, the invention con- 
templates many system configurations and is not limited to 
a particular system configuration or component segmenta- 
tion scenario. All that is required of a given component 
configuration is adequate intra -system component intercon- 10 
nection that supports operability and cooperation of the 
components to produce a viable displayed image. Whatever 
component and housing configuration is used, the housing 
preferably is compact and light-weight, and preferably con- 
sists of strong, rugged material that is impervious to the 15 
environment, e.g., by being water resistant. The housing 
should also preferably provide good shielding from atmo- 
spheric electrical noise. These various housing characteris- 
tics are provided by, e.g., conventional military-grade hous- 
ing materials. Electrical wiring, optical fiber, and other 2 q 
interconnections are also preferably shielded by suitable 
cabling to render the interconnections impervious to the 
environment. 

A wide range of display hardware is contemplated by the 
invention. The real time digital video images produced by 2 s 
the imaging system are suitable for any display system that 
is compatible with standard electronic image data. For 
example, grayscale monitors, color monitors, cathode ray 
tubes, solid-state displays such as liquid crystal, 
electroluminescent, or other solid-state displays, projection 30 
displays, microelectromechanical displays, or other suitable 
displays, of any size, can be employed in the imaging 
system. In any case, the display can be located near and 
possibly in alignment with the optical input and imager 
components of the imaging system, as in a hand-held, 35 
head-mounted, goggle, or weapon sight configuration. 
Alternatively, as discussed above, the display can be located 
remotely from the imager and possibly also remotely from 
the other system components. Whatever spatial configura- 
tion is selected, any number of local and any number of 40 
remote displays can be connected in parallel for simulta- 
neous and parallel viewing on multiple displays. 

Turning now to a detailed discussion of the other com- 
ponents provided in the imaging system of the invention, the 
first of those components, the optical input 14 (FIG. 1) is 45 
embodied as a suitable transparent window or lens based on 
a given optical specification for the system and a given 
housing configuration. A lens input can consist of a planar or 
curved geometry or multi-element lens system. For example, 
in the imaging systems described above, the optical input 50 
preferably consists of a high-quality lens with a focal length 
that is sufficient to accommodate a field of view of interest. 
An example lens suitable for the imaging system is charac- 
terized by a focal length of about 25 mm, a T-number of 
about 1.35, subtending about a 40 degree central field of 55 
view. 

The field of view of the imaging system of the invention 
is set by the field of view of the optical input of the system. 
For example, a relatively wide field of view, e.g., 100 
degrees of view, can be obtained with a corresponding 100 60 
degree field of view simple lens of suitable focal length. 
Although a wide field of view simple lens compromises 
various performance aspects of the imaging system and may 
require a multi-element lens system to correct for any field 
distortions that are incurred by the simple lens, such a lens 65 
can be used with the system if necessary. It is preferred, 
however, that a curved, short focal length lens be employed 
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for enabling imaging of a wide field of view. An embodi- 
ment of the imaging system of the invention tailored to a 
wide field of view application through use of a curved lens 
and curved imager will be described in detail later in the 
discussion. 

Referring back to FIG. 1, the ambient input light acquired 
by the imaging system optical input port 14 is directed to the 
one or more imagers 16. Each imager preferably consists of 
a CCD that is a thinned, back-illuminated, multi-ported 
frame transfer device employing narrow-trough buried 
channel MOS structures and buried contact interconnec- 
tions. This CCD configuration, as provided by the invention 
and in conjunction with the analog signal processor 
described below, enables the imaging system of the inven- 
tion to achieve significantly better low-light-level imaging 
performance than conventional electro -optic intensifier 
tube-based imaging systems. The imaging system CCD 
provides a peak quantum efficiency greater than about 90%, 
imaging speeds compatible with real time video sequence 
speeds of about 30 frames/second, broad spectral sensitivity 
of about 0.3 //m-1.1 /^m, high spatial resolution, and optical 
sensitivity even in overcast starlight. Importantly, the imag- 
ing system CCD enables viable low-light-level imaging with 
an enormous intra -scene dynamic range, e.g. of about 
60,000 grayscale levels for a 16 bit digital image signal. The 
imaging system CCD also provides electronic shuttering and 
anti-blooming capability as well as near-ideal modulation 
transfer function characteristics. 

As shown in FIG. 3A, the CCD imager 34 includes an 
image area 37 of imaging pixels and an adjacent frame-store 
area 38 of frame charge storage pixels. Imaging pixels in the 
imaging area 37 and frame storage pixels in the frame-store 
area 38 are aligned in rectangular arrays as is conventional; 
in an example of a particularly advantageous CCD imager 
configuration in accordance with the invention, the imaging 
array and frame-store array each include a 640x480 array of 
pixels that are segmented into four vertical regions, one for 
each of four outputs ports, as described below. It is the CCD, 
imaging array 37 to which ambient input light is directed by 
the imaging system optical input; the imaging array 37 is 
thus preferably in corresponding and unobstructed align- 
ment with the optical input port. The frame storage array is 
covered by an opaque light shield, e.g., a layer of metal, as 
described below, to prevent input light from impinging the 
frame storage array 38. A number of extra rows of pixels (not 
shown), for example, four extra pixel rows, are preferably 
included between the imaging and frame storage arrays to 
compensate for any lateral alignment mismatch between the 
light shield and the underlying frame storage pixel array. A 
number of extra columns of pixels (not shown), for example, 
two extra pixel columns, are preferably included on each 
side of the imaging array to collect edge-generated dark 
current, i.e., current generated due to ambient thermal con- 
ditions rather than an optical input. 

Briefly describing the general operation of the CCD 
imager, photons that enter the low-light-level imaging sys- 
tem through the optical input port and are incident on pixels 
of the imaging pixel array generate packets of 
photoelectrons, in the silicon of each image array pixel, that 
are collected in a conventional manner based on the three 
clocked voltages. After a period of time provided for inte- 
gration of the collected photoelectron charge packets, the 
charge packets are rapidly transferred from the imaging 
pixel array columns to the frame-store pixel array columns 
in parallel. Then during a next sequential integration period, 
during which incident light generates fresh photoelectron 
charge packets, charge packets already in the frame storage 
pixel array are transferred into output registers, described 
below. 
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The extra rows of pixels preferably provided between the n-channel CCD imager, the blooming drains are heavily 

imaging and frame storage pixel array compensate for doped n-type regions. The blooming barriers are regions of 

so-called transfer smear that can occur as charge packets are doping different from that of the. buried channel 70, and the 

transferred from the imaging array to the frame storage barrier doping is selected based on a desired mode of 

array. Transfer smear occurs when the charge packets are 5 operation, as explained below. 

transferred through a pixel located in correspondence with a j?jg. 4B, a cross section taken along A-A' in FIG. 4A, 

very bnght region n .the ^image area on their way to the ^ ^ 4U ti , he fc nase , 

frame storage array. The bright region generates additional ^^^ta^^a^^jo^b^iSaa 

charge so-caUed transfer smear charge, in the charge pack- ^ buried ^ J^l m ^ the 

ets of all pixeb moving through the given pixel. The exfra 10 ^ , a heavil t re ^ on 420 is provi ded 

rows of pixels enables the transfer smear charge to be , r . . ■ . • j u - u L: i„ 

. . J . . , , . ° . . underneath the blooming drains and blooming bamers. In 

sampled from each pixe column thereby providing the ^ * dcflect hotodectrons e - 425 

abihty to detect and cancel the transfer smear in later signal from £ y £ m ^ 6 back side of the substrate 

processing stages. toward ^ bufied channel In the of this barrier the 

Both the imaging pixels and frame storage pixels prefer- 15 blooming drains could collect a s i gD i fic ant number of pho- 

ably are each defined by three adjacent conducting gate toelectK)ns; lhis of counje fe undesirable because it would 

structures, e.g., a polysilicon gate structures, as is conven- feduce the quantum efficiency of the imager, 

tionally provided in a so-called metal-oxide-semiconductor ... in . . . ,. 4 t . 

(MOS) structure, in a three-phase configuration that is FIG. 4C is a potenUal diagram corresponding to the cross 

controlled by three clocked voltages, <fr 2 , and <f> 3 , in the 20 ^ ctl0nal view of FIG. 4B for illustrating the operation of he 

usual manner. An example pixel confi^iration for a CCD b oomm S drains and blooming barriers In operation, the 

imager in accordance with the invention is shown in FIG. blo ?T g * "\ /^h ^T* ^Vi*' ifT 

3B. The pixel 40 is defined by three adjacent MOS vo ^ a given pixel gate voltage, the potential well in the 

structures/like the left-most structure 42, each MOS struc- s » bstra * ^ * **P^ P°f ve > m the *™f 

ture including a gate electrode for connection to a corre- 25 chani * 1 re f>° /° than * the bloomm g barr / er ^^410, 

sponding phase voltage, fc, and <t> 3 , for controlling the wbereb y tbe bl ™ mm * Earners serve as channel stops In 

potential well in the sHicon substrain 44 under the gate wor ^> the blooming barners define the walls of the 

electrodes. Preferably, the substrate region is adapted, as char S c P ackct P° tential weU m the buned channeL 

described in the fabrication sequence below, to provide a As the buried channel potential well fills with charge, a 

buried channel region, rather than a surface channel region, 30 P oint can be reached at which the well is effectively "full" 

in which charge packets are held in the MOS structures. and n0 additional charge can be stored in the well; in this 

Also, the gate electrodes are preferably insulated from the ca se, as shown in FIG. 4C, the excess charge flows over the 

substrate by a layer of silicon dioxide 46 and a layer of blooming barriers 410 and into the blooming drain 400. 

silicon nitride 48, as described in the fabrication sequence Connections to the blooming drains at the periphery of the 

below. 35 imager provide a means for collecting this excess charge. 

Preferably, the buried channel regions of the pixels are Thus, the dooming drain configuration provides both chan- 
isolated by channel stop implant regions that run parallel to nel sto P and anti-blooming functionality. As will be recog- 
the direction of charge flow in the buried channels and define nized b X those sldlled in art, other channel stop and 
the edges of the channels. In one suitable configuration in anti-blooming configurations are also suitable, 
accordance with the invention, the channel stop regions are 40 Turning back to the pixel configuration, the pixel dimen- 
defined by a configuration that also provides blooming sions can be of any suitable size, with the caveat that smaller 
control of the imaging array pixels. Blooming is the well- sizes limit the size of charge packets produced by a given 
known phenomenon in which an image of a bright object, pixel. In one example, the image pixels are about 12.7 
under pixel overload conditions, results in an apparent size fiimxll.l /ma, and the rows of frame storage pixels decrease 
increase of the object in the resulting image. This is a result 45 m DOtD horizontal and vertical dimensions, say by about 
of accumulation of an amount of charge in a given pixel that 10% smaller; this is schematically shown in FIG. 3A, where 
exceeds the charge packet well capacity of the buried the two left-most frame storage array segments branch out 
channel of the pixel. When this occurs, the charge in excess from the two right-most frame storage array segments due to 
of the well capacity spreads into adjacent pixel wells, increasingly smaller frame storage pixel sizes. This shrink- 
thereby increasing the apparent size of the bright object that 50 of tne &ame storage array pixels provides especially 
produced the excess charge. In severe cases, blooming can good accommodation of four output registers and ports, as 
affect a large fraction or even all of an image, making it discussed below. Other pixels sizes and output porting 
impossible to view faint imagery in the same scene. Bloom- schemes are within the scope of the invention, however, and 
ing control is thus important for enabling viable imaging of no particular pixel sizing or output porting configuration is 
a low-light-level scene that includes a wide intra-scene 55 required. 

dynamic range. Returning to the CCD imager configuration, in one 

Referring to FIGS. 4 A, 4B, and 4C, there is shown a pixel example configuration, as shown in FIG. 3A, four output 

blooming control configuration employed in the invention to registers 39a, 39b, 39c, and 39<i, are employed. A portion of 

also provide channel stop functionality. This particular con- the charge packets in a full width of each frame-store pixel 

figuration accommodates back side illumination of the 60 row is loaded into each register during one output transfer, 

pixels, which as mentioned above, is the preferable mode of based on the number of stages provided in each register. In 

imager illumination. Details of the back side illumination the example scenario given above of a 640x480 image pixel 

mode will be presented below. FIG. 4A shows two MOS array size, each register includes 160 stages and thus each 

gates, and ^ of a three-phase pixel on the front side of vertical pixel array segment includes 160 charge packets, 

a substrate. In place of conventional channel stops, bloom- 65 160 pixel-wide packet rows in each segment are therefore 

ing drains 400, each flanked on both sides by blooming transferred into the corresponding output register at one 

barriers 410 are provided. In an example case of an time. As explained in detail below, the design of the registers 
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includes several specific features to limit the amount of same silicon substrate as the pixel arrays. The charge-to- 
noise generated during the charge transfer process. voltage conversion process proceeds as follows. A charge 
Each of the registers comprises a row of gates in align- P«*e» Q.» transferred through the register stages; the last 
ment with the last row of frame storage pixel gates in the * ree re Sf 8 ate sta 8 es 52a > 52c > are ahw * n , m . tbe 
pixel array segment just above a given register A row of 5 figure. The ast register stage gate provides an output ton 
K * - i L i.ii j 11 the register to the sense node 53 of the circuit, which has a 
frame storage pixel charge packete clocked vertically into a J c 54 ^ nodc 53 fe also COQnectcd t0 the 

given serial register is then shifted horizontally to output the ^ 5fi of an ^ fidd effect transistor (FFr) 58 prefer . 

charge packets in sequence. In a preferable design, the ably operated in a source-foUower mode. The charge packet 

so-called configuration of binning is accommodated by the Q delivered to me capac i tanc e 54 produces a voltage cor- 
output registers. Vertical binning entails accumulation of 10 respondillg t0 the amount of charge in the packe t, thereby 

two or more charge packets from adjacent pixel rows in a producing an analog output signa i 6 0 at a node 62 of the FET 

given pixel column in each register stage to form, e.g., a ^ rorresponds t0 the ^put charge packet. The output 

double-sized register stage charge packet before the charge s{ { 6Q ^ theQ dclivered t0 thc analog signal processor of 

packets are delivered serially to the output circuit; this ^ j magmg S y S tem 

effectively sums the two or more pixel rows in the charge M ^ an ^ ^ ^ ^ ^ for a 

domain. To enable this operation the renter preferably rf ^ ^ ^ fmm ^ ^ 

consists of a buried channel region that is twice or more the r . eA f . P ^ ^ ( . • 

. i . . <■ -i capacitance 54 to reset the capacitor tor the next incoming 

channel width of that of the image and frame storage pixels. ^ ^ controUed b a reset FEX 64> the 

The binning operaUon is inherently free of noise generation Vwhich is controUed by a clocked reset voltage, *„, 

because the summing occurs in the charge domain, and zu , t , , - c . . , . # , . ^ , u • „ mU a 

, . *f j « i and the drain of which is connected to a dc bias voltage V^. 

provides the ability to greatly increase the frame speed ot the _ t . ,,*<•■ r i . • . . 

CCD imaeer design and fabrication of the output circuit, output 

, . registers, and pixel arrays of the CCD imager are customized 

As will be understood by those skilled in the art, hon- to si ificantly ^ me noise and boost the imaging per _ 
zontal binning can also be accommodated to sum adjacent formance G f the CCD imager. Two primary results of the 
pixel charge packets in a given pixel row of a register. In this deg . features m ^ h quantum effic j e ncy and very 
case, no special structure is needed to sum the charge i ow output signal noise; these two conditions enable opera- 
packets; instead, the output circuit, described below, is tion of tfae j. m of the invention at significantly 
controUed such that two or more adjacent charge packets are ^ ved per f ormance levels and frame speeds compared to 
dumped onto the output node of the register before the conventiona i low-light-level imaging systems. In a first of 
output circuit accepts the charge. In a more flexible and in ^ cust0 mizations, men tioned previously, all of the pixel 
some cases preferable method of horizontal burning, a MOS structures and output circuit FETs are fabricated with 
summing gate is provided between the last serial register a buried chanflel A bufied channelj ^ oppQsed tQ a ^ 
stage and the output gate. The summing gate is biased to ventionaI surface channelj can be biased by tne gate elec- 
collect two or more consecutive charge packets transferred imo an electronic potential regime where me current 
to it by the serial register. After the last charge packet is flowi thlou ^ the chaQnel fe k awa from the 
summed, the summing gate is clocked to deliver the summed ^ Umits lhe amount of in of CQ carriers ^ the 
charge packet past the output gate to the output circuit. channel due tQ imeraclion of the carriers ^ surface states 

Horizontal and vertical binning of an image array can be D f the substrate in which the channel is formed. The buried 
employed to improve the resolution capability of the imager ^ channel configuration is also exploited to allow for spacing 

under low light level conditions in which the resolution is G f the gate away from tbe drain in the FETs to limit 

limited by the noise of the system rather than the geometry gate-drain capacitance, as described in detail below, 

of the pixels. Even with a double-width register channel for In a Gjtthci enhancement, an additional implant step, 

accommodating vertical binning, the CCD imager can be described below, can be employed in the fabrication 
operated in a normal mode without the charge summing 4S sequence to define a narrow trough region in the center of 

process. the buried channel along its length to further limit the 

Turning to output operation of the imager, charge packets exposure of a charge packet to the surrounding silicon 

in a given register stage are clocked, e.g., in a three-phase substrate. In this case, the trough implant preferably consists 

manner, out of the register and directly to an output circuit of a narrow implant of the buried channel species, thereby 
located on the substrate with the image pixel array and frame 50 producing a narrow charge confinement channel. This 

storage pixel array. This is accomplished using an output trough provides a deeper potential well than the surrounding 

diode, as shown in FIG. 3C, which illustrates the three-phase channel region, resulting in a tendency for the charge to 

register stages at the end of a register. The buried channel 70, spatially remain in the area of the trough. In this 

defined by a dopant implant as described below, is aligned configuration, the probability of encounters between the 
with an output diode 72 formed of an additional dopant 55 photoelectronic charges and trap sites in the substrate is 

region of the same dopant type to form a p/n junction at a reduced. This is particularly beneficial for very small charge 

point beyond the last serial register stage. A metal intercon- packets that without the channel trough would tend to spread 

nection 74 makes contact to the p/n junction region for across the full width of the channel and generate trap-related 

transferring the charge to a sense node of the output circuit, noise charge. 

described below. At the end of the row, a channel stop 60 [ n a ne xt preferred design configuration, the image pixels 

implant 76 and field oxide region 78 provide isolation of the are provided with electronic shutter mechanisms that elimi- 

buried channel. nate image degradation caused by smearing of an acquired 

Referring also to FIG. 5, an output circuit 50 at the output image, in which, as explained previously, charge packets 

of each output register converts tbe transferred charge to a acquire additional charge as they are transferred through one 
voltage, thereby producing four parallel analog electronic 65 or more pixels corresponding to a very bright region of an 

pixel signals that represent the light input to the imaging image on their way from the imaging array to the frame 

system. This output circuit is preferably fabricated on the storage array. The shutter mechanism also enables resolution 
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of incident optical pulses that are closely spaced in time. Ia 
addition, the electronic shutter can provide real time acqui- 
sition of images of fast-moving objects, with reduced 
blurring, by enabling a very short exposure time. 

Electronic shutters are employed in the CCD imager of 5 
the invention to effectively close off the image pixel array to 
the collection of photoelectrons generated by the incident 
light during a portion or the entire duration of the time 
required to transfer all of the charge packets from the image 
pixel array to the frame storage pixel array. This limits the 10 
amount of charge that is integrated into the image pixel 
charge packets to eliminate the possibility of smearing of an 
image and to enable very high speed image capture. 

Referring to FIGS. 6 A and 6B, two structures are 
employed in the invention to provide electronic shuttering of 15 
the image pixels. These structures are based on the back 
side-illuminated CCD mode mentioned previously. 

As shown in FIGS. 6 A and 6B, the three-phase pixel 
includes, e.g., an n-type buried channel 70 over which the 2Q 
three phase gate electrodes, designated as ty l7 (|> 2 , and <t> 3 run 
in a perpendicular fashion. The additional shutter structures 
include an n + shutter drain 82a, 826, on each side of the 
buried channel, and a stepped p-type buried layer 84 extend- 
ing across the width of the pixel. Id FIG. 6 A, the three 1$ 
relatively shallow regions of the buried layer are designated 
as 84a and the two relatively deep regions of the buried layer 
are designated as 846. Blooming barrier channel stops 86 are 
also shown; these define the extent of one pixel along the 
three gates. Shutter drain electrodes are preferably provided 3Q 
to enable setting the shutter drains to a desired potential. A 
shutter drain connection to the CCD imager is thereby 
required, not shown in the example of FIG. 6A. 

In operation, to open the shutter of the CCD image pixel 
array during one of the three clock phase periods, a voltage 35 
is applied to the corresponding phase gate electrode such 
that a depletion region extends from the surface through the 
buried channel 70 and shallow region of the buried layer 84 
into the substrate. At the same time, a voltage is applied to 
the shutter drain 82 to reverse bias the shutter drain/substrate ^ 
junction, but is held low enough such that there is no 
depletion through the buried layer under the shutter drain. 
Photoelectrons generated by the incident light in the sub- 
strate are attracted by the depletion region electric field to 
the buried channel, toward which they enter through the 45 
shallow buried layer region, then being stored in the chan- 
nel. The undepleted shallow region of the buried layer 
creates a potential barrier that repels the photoelectrons 
away from the shutter drain. A similar potential barrier stops 
the photoelectrons from entering the buried channel at deep 50 
buried layer regions. 

The potential well formed in the buried channel adjacent 
to the deep buried layer is more positive than that in the 
channel adjacent to the shallow buried layer. As a result, the 
photoelectrons that enter through the shallow buried layer 55 
are transferred to the buried channel region above the deep 
buried layer. The depletion region under the central part of 
the channel, where a trough, as described above, can be 
incorporated, is therefore insensitive to the number of pho- 
toelectrons that have been captured up to some maximum 50 
amount. Additional photoelectrons beyond this amount are 
shared between the central and outer regions of the pixel 
channel, and cause a gradual collapse of the depletion 
region. 

When the electronic shutter is closed, the voltage applied 65 
to a given pixel gate electrode is not sufficient to deplete the 
substrate through the buried layers, but is adequate to 
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transfer charge from one pixel to the next. A large voltage is 
applied to the shutter drain 82 to cause a depletion region to 
spread beyond the shallow buried layer and deep into the 
substrate. Photoelectrons generated by incident light in the 
substrate are attracted by the electric field of this deep 
depletion region, and move toward the shutter drain rather 
than the buried channel. As a result, no substantial charge is 
additionally acquired in the channel. 

The stepped buried layer of the electronic shutter structure 
effectively creates a central channel collection region and 
side storage regions in the channel. This provides a depletion 
region that is independent of the signal accumulated in the 
channel well up to some maximum amount, whereby the 
response of a given pixel is linearly dependent on integration 
time. Also, at high signal levels, the shutter devices provide 
a sublinear integration response, thereby increasing the 
dynamic range of the pixel. Other features and advantages of 
the electronic shutter design are provided by Reich et al, in 
U.S. Pat. No. 5,270,558, issued Dec. 14, 1993, the entirety 
of which is hereby incorporated by reference. 

As explained above, the CCD imager is preferably con- 
figured such that the back side of the substrate, rather than 
the front side on which the pixel MOS gates and intercon- 
nections are supported, is that side on which input light to be 
imaged is incident. In this configuration, incident light 
generates photoelectrons toward the back side of the 
substrate, the photoelectrons then migrate toward the pixel 
buried channel under the influence of the channel depletion 
region electric field. It is thus preferable in many cases that 
the CCD imaging substrate be thinned to minimize recom- 
bination as well as lateral spreading of the photoelectrons as 
they traverse the wafer. A fabrication process for accommo- 
dating back-side illumination of a thinned CCD structure 
will be described later in the discussion. 

The main advantages attained by the back-side illumina- 
tion scheme are extremely high quantum efficiency, very 
broad spectral response, excellent spatial uniformity, and 
superior inter-pixel and intra-pixel response. Indeed, the 
quantum efficiency of a back-side illumination imager in 
accordance with the invention is almost reflection limited 
throughout the visible spectrum and is highly uniform across 
a pixel array. In the back-side illumination configuration, 
incident light impinges a uniform, substantially flat substrate 
field, rather than the cluttered area on the front surface of the 
substrate. Front-side illuminated CCD imagers are limited 
by, among other things, absorption of incident light by the 
pixel gate electrodes and other interconnections; this absorp- 
tion is entirely eliminated in the backside scheme. Optical 
absorption is particularly important for imaging in the UV 
and short visible wavelengths, all of which are characterized 
by very short absorption lengths. The back-side illuminated 
CCD imager achieves a broad imaging spectrum by elimi- 
nating this optical absorption. 

In addition, the substantially flat imaging surface attained 
by the back-side illumination configuration enables custom 
matching to a desired imaging wavelength by deposition of 
a corresponding anti-reflection coating on the back surface. 
Furthermore, the modulation transfer function of the back- 
side illuminated imager is more benign than that of a 
front-illuminated imager. The response of a front- 
illuminated device to a point source is known to be very 
rough; this roughness is caused by the overlapping pixel 
gating structures. Wherever the gates overlap, the incident 
light is attenuated upon passing through the structure by as 
much as a factor of two over that passing through a non- 
overlapping gate region. In contrast, the back-side illumi- 
nated imager provides a smooth, unobstructed entry surface 
over the entire span of the imaging pixel array. 
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It is important to note that in a back-side illumination 
scheme, the back substrate surface must be passivated to 
stabilize any surface states present at that surface. If the 
surface is not passivated, an electric field can be produced by 
trapped charges in the native oxide that is inherently present 
on the substrate; this could result in acceleration of gener- 
ated photoelectrons to the back surface, where they could 
recombine with the trapped charge, rather than moving 
toward the pixel channel. Passivation of the back surface 
also reduces generation of parasitic dark current in the 
imaging pixels. The CCD imager fabrication sequence pre- 
sented below will discuss additional considerations of the 
back-side illumination configuration. 

Beyond passivation of the back surface of the substrate, 
various pixel configurations can be employed to further limit 
dark current generation in the CCD imager. One such 
configuration employs a three-phase pixel operation in 
which a charge packet stored in the potential well of a given 
pixel's buried channel is transferred and retransferred under 
the three phase gates at a rate that prevent dark current from 
being generated under any one of the phase gates. This 
technique is described by Burke in U.S. Pat. No. 5,008,758, 
issued Apr. 16, 1991, the entirety of which is hereby 
incorporated by reference. 

Although a back-illuminated CCD imager is preferred in 
the invention, it is possible to achieve high sensitivity that 
can approach, but not typically equal, that of the back- 
illuminated configuration using a front-illuminated configu- 
ration in which there is provided a localized open region of 
each pixel on the front of the substrate that is kept free of 
overlying material, using a suitable gate design. In such a 
case, only that fraction of the pixel comprising the open 
region has high photosensitivity; the fill factor is thus less 
than unity, typically on the order of about 30%. While in the 
open area the sensitivity can be equivalent to that of the 
back-illuminated configuration, the over-all sensitivity obvi- 
ously is reduced by the less-than-unity fill factor. The 
effective fill factor can be improved, by about a factor of 
two, through the use of an array of micro-lenses, one for 
each pixel, arranged so as to concentrate the light incident 
onto each pixel principally into the open area of the pixel. 
Through the use of these techniques, the sensitivity of the 
front-illuminated imager can be made to approach that of the 
back-illuminated imager to within about 60%. 

The design and layout of the output circuit (FIG. 5) of the 
CCD imager is also customized to limit noise and boost 
overall operational performance of the low-light-level 
imager. To a large degree, the minimum attainable noise of 
the overall imaging system is limited by the amount of noise 
generated with the analog output signal of the CCD output 
circuit. Indeed, it is well accepted that the output circuit for 
a given CCD system sets the minimum noise level generated 
by the overall system. Several features are provided by the 
CCD imager of the invention to limit the amount of noise 
generated by the circuit. First, as with the MOS structures of 
the CCD imaging and charge storage pixels, the output 
circuit FETs (56, 64 of FIG. 5) are preferably fabricated to 
provide a buried channel rather than surface channel. As 
shown in the example output FET of FIG. 7 A, a buried 
channel 66 is provided under the gate region. As stated 
above, this configuration limits the noise generated by the 
FET. 

In the CCD imager of the invention, noise generated by 
the output circuit is dramatically limited by a combination of 
design features that reduce the capacitance of the circuit. In 
general, as the capacitance of the circuit is decreased, the 
corresponding voltage generated by the output FET is 
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increased, because the magnitude of charge packets dumped 
into the output circuit is directly proportional to the product 
of the circuit capacitance and developed voltage. Thus, a 
lower capacitance circuit produces a larger voltage change 
5 for a given number of electrons in a charge packet; in the 
output circuit of the invention, this raises the output signal 
voltage above the inherent noise voltage of the source- 
follower FET. 

The dramatically low noise level of the CCD imager of 
10 the invention, in combination with the high quantum effi- 
ciency achieved with the back-side illumination 
configuration, enable the imager to produce a viable low- 
light-level image without the use of a gain mechanism to 
first amplify low-level input light. With this low-level imag- 
15 ing capability, the imaging system of the invention can 
eliminate the various mechanisms known to limit the per- 
formance of conventional low-light-level imagers such as 
conventional intensifier tube imagers and intensified -CCD 
imagers. 

20 Referring again to FIG. 7, the output FET of the output 
circuit is preferably designed to minimize the circuit's 
capacitances, including the drain-to-gate capacitance. In the 
invention, the edge of the drain region 68 in the substrate is 
separated from the edge of the gate 80 by a minimum offset 

25 distance, O, to decrease the drain-to-gate capacitance. Mea- 
surements made on various test structures have shown that 
the drain-to-gate capacitance continually decreases as the 
drain is moved out to about 1.5 /an away from the edge of 
the gate and remains constant as the drain is moved further 

30 away. Thus, it is preferable that the drain offset distance, O, 
be about 1.5 /mi. This offset has been verified experimentally 
to result in an increase in the responsivity of the charge-to- 
voltage conversion of the output circuit by a factor of about 
1.5 to 2. Similarly, the source-to-gate capacitance can pref- 

3 5 erably be minimized by separating the edge of the source 
from the edge of the gate by a distance of about 1.0 /mi. 

The lateral area of the output FET gate 80 is also 
preferably selected to increase the responsivity of the circuit. 
Additionally, it was found experimentally that the noise 

40 generated by the output circuit continually decreases as the 
gate length is decreased from about 4 /mi to about 2 /mi for 
a given gate width. Although the circuit responsivity con- 
tinues to increase as the gate length is further decreased, the 
noise level does not at that point further improve. Thus, an 

45 output FET gate length of about 2 /mo is preferred. In terms 
of output FET gate area, it was found experimentally that for 
high frequency low-light-level imaging, e.g., at frequencies 
greater than around a megahertz, a 10/2 ratio is preferred. 
For lower frequencies, no particular gate area ratio is pre- 

50 ferred. 

Referring again to FIG. 5, the sense capacitor 54 
employed by the output circuit for generating a voltage in 
response to a charge packet is preferably designed to mini- 
mize the area and corresponding parasitic capacitance of the 

55 capacitor. Such capacitance is in general inherent to the 
monolithic diffusion configuration of the capacitor. In a first 
of two primary techniques for reducing the parasitic 
capacitance, the contact to the p/n junction at the output of 
each CCD register is preferably a buried, rather than con- 

60 ventional butt, contact. As shown in FIG. 8, which is a 
slightly different rendering of the view of FIG. 3C, showing 
the output connection between a CCD register and an output 
circuit, the buried contact configuration enables a compact 
and correspondingly low capacitance connection between 

65 the p/n junction charge collection port 72 and the intercon- 
nection 74 that leads over the field oxide region 78 to the 
sense node and gate of the output FET, as shown in FIG. 5. 
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In an example configuration in which the buried channel channel region is thereby maximized, while at the same time 

70 of the register is n-type and the substrate is p-type, the charge is efficiently "funneled" to the charge-collection 

diode collection port is preferably an n + -type region, which junction. Similarly, the buried channel 83 of the output 

forms an n"7p junction with the substrate. In a preferable circuit reset FET (64 in FIG. 5) is limited to a funnel-shape 

technique for forming the buried contact, the n + region and 5 reg ion 87 between the edge of the reset FET gate 88 and the 

its connection to the interconnection 74, e.g., a polysilicon charge-collection node 72 such that the parasitic capacitance 

interconnection that is also the output FET gate, are formed associated with the channel stop in the area of the reset 

simultaneously and in a self-aligned manner. The resulting channel is minimized 

structure occupies less area than a conventional butt contact A » * . ' . * ^ 

■m . ~i j • • -i -*u *u As a result or the various noise reduction ieatures pro- 

and significantly reduces the capacitance associated with the 1P1 " : JrJ; ~: vailuuo . ^ . *\ 

contact vided by the CCD imager of the invention, a dramatically 

' . , c . . - , . . . vu high signal-to-noise ratio is attained for the overall low- 

Tht buned contact configurat.on is fabncated with an fa ^ Fof , ^ t6chni 

elegantly simple prcce^ sequence that is integrated into the ^ for minimizin ^i,^ kad t0 va i ues as 

overall CCD process discussed be ow, but is pointed out , ow about 1Q femtoFarads and ferabl as low as about 

here for clarity. Cohering formation of the p/n junction 1S femtoFarads for the itor (FIG ^ of ^ out t 

jnterconnect with the first polysihcon layer of the three c ^ ^ ^ ^ tQ values Qf ch to . volt 

layers employed in the three-phase CCD design, the contact conversi j the vol , duced at the output 60 of the 

is formed after sihcon dio»de and I silicon nitnde layers have circuit ef electmQ , ransferred from ^ out t 

been formed on the front side of the CCD substrate to act as ^ , o ^ ^ nod Qf ^ 2Q microvolts electron . 

a gate insulator. First, the sdicon dioxide and silicon mtnde 20 M operation / frequencies of about SO kHz, the CCD imager 

layers are patterned and etched to expose the substrate in an Qoise „ a level rf onl about ^ 5 electrons and 

area corresponding to the lateral geometry of the charge- ^ ational fancies, e.g., at about 4 MHz, the 

collection junction. Then a polysihcon layer is deposited to . °7 , . * , n . . - , , 

? . * . r , ; ' , r 4 . noise level is only about 4-7 electrons, 

form the interconnection. It is preferable that any native J . 

oxide on the substrate be removed prior to the polysilicon « Tum ing now to fabrication of a three-phase CCD imager 

deposition. The polysilicon can be deposited in a polycrys- for the low-light- level imaging system of the invention one 

talline state and then doped to render it highly conducting, exam P le fabrication sequence is described below. As will be 

or alternatively, can be deposited in a quasi-amorphous state recognized by those skilled in the art, many variations of the 

and doped in situ during the deposition. P rooess parameters can be employed; the invention is not 

Due to its morphology, the polysilicon layer enables the 30 t0 * SpcdfiC fabrication or s P ecmc COndl " 

dopant atoms to diffuse quickly through the layer and then hons tor the ^W™™- 

to diffuse into the silicon substrate in the area of the contact In one example fabrication process in accordance with the 

opening in the gate insulator layer. Because the silicon invention, pMype silicon substrates, of about 0.01-0.02 

dioxide-silicon nitride insulating layer is relatively resistant Q ^ and havm S a P" tv P e epitaxial layer of about 30-50 

to dopant diffusion, substantially no dopant diffuses through 3S ^rn, are employed as a starting material. This p epitaxial 

the insulator, resulting in a relatively anisotropic diffusion la y er ultimately forms the back-illuminated CCD substrate 

into the substrate. In a final step, the polysilicon layer is after the CCD MOS and FET structures are formed. The 

patterned and etched to define the gate area of the output substrates are cleaned at the start of the process and through- 

FET This simple process results in an effective charge- out the process in the conventional manner, 

collection junction and very compact interconnection that 40 The gate insulator is first formed by growing a silicon 

minimizes capacitance associated with the interconnection. dioxide layer (46 in FIG. 3B) in a dry growth process at 

The parasitic capacitance of the charge -collection junc- about 1000° C. to produce an oxide thickness of about 600 

tion is also a function of the distance from the buried channel A. Then a layer of silicon nitride 48 is deposited by, e.g., 

to the channel stop implant 76, as shown in FIG. 8. As shown chemical vapor deposition at a temperature of about 788° C. 

in the figure, it is preferable that the buried channel 70 not 45 10 produce a nitride thickness of about 400 A. The channel 

extend beyond the location of the charge-collection junction stop regions are then photolithographically defined and the 

72 such that the parasitic capacitance C between the channel exposed silicon nitride is etched using a conventional 

stop layer 76 and the region around the junction is mini- plasma etch employing, e.g., an SF^Oj chemistry, 

mized. In conventional CCD designs, as shown in FIG. 3C, At this point, a boron field implant of about 100 keV a 

the buried channel 70 is produced by a blanket ion implan- 50 dose of about 7xl0 12 /cm 2 is undertaken to define a channel 

tation step into all channel regions, including the areas 89 stop field surrounding the channel area of the MOS struc- 

between the charge-collection junction and the field oxide hires and output circuit FETs. After the boron implant, the 

78 overlying the channel stop 76. In this configuration, a channels are isolated by a conventional local oxidation 

significant parasitic capacitance is developed between the process in which wet oxidation at about 1000° C. is carried 

channel and the channel stop regions. 55 out to produce a 5000 A-thick oxide layer over the channel 

In the invention, this parasitic capacitance is reduced by stop regions, while the existing oxide/nitride in the channel 

restricting the location of the buried channel to a limited regions remains largely unchanged, 

region within the channel area such that charge is effectively A scupper for removing perimeter dark current generation 

funneled to the charge-collection junction but the distance of is then produced. A channel region in the form of a narrow 

the buried channel to the channel stop regions is maximized 60 stripe running around the perimeter of the imager is reserved 

wherever possible. One example of this configuration is for the scupper. The scupper is photolithographically 

shown in FIG. 9. Here the buried channel 70, extending defined; and then a phosphorus implant at about 150 keV 

under the gate 81 of the last stage of an output register, is and a dose of about lxl0 14 /cm 2 is carried out to form an 

funnel-shaped in the region 71 between the edge of the gate n-type region that in operation is biased to a positive 

and the charge-collection junction 72 underlying the poly- 65 potential to intercept and collect dark current and photoge- 

silicon interconnection 74. The distance from the buried nerated charge from the periphery of the device before such 

channel to the channel stop region 76 surrounding the charge can contaminate the edges of the image. 
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At this point, the buried channel is formed by first 
depositing and patterning a layer of photoresist in areas 
around the buried channel location. Phosphorous is then 
ion-implanted at about 190 keV and a dose of about lxlO 12 / 
cm 2 ; the resist is then removed. Photoresist is next deposited 5 
and patterned to define the buried channel trough, and 
phosphorus implanted at these regions at about 190 keV and 
a dose of about 3xl0 1:l /cm 2 . 

Three layers of polysilicon are next deposited and pat- 
terned to produce the three-phase pixel gate configuration. 1Q 
Each polysilicon deposition is conducted by chemical vapor 
deposition at about 580° C, the first deposition carried out 
to produce a layer of about 4200 A in thickness, the second 
carried out to produce a layer of about 4000 A in thickness, 
and the third carried out to produce a layer of about 3300 A 
in thickness. In all three deposition processes, the film is 15 
doped with phosphorus in situ during the deposition. After 
each deposition process, the films are annealed in nitrogen 
at a temperature of about 800° C. for about 30 minutes. Then 
the upper layer of phosphosilicate glass that forms during 
the anneal is removed with a buffered oxide etch. 20 

Each polysilicon layer is photolithographically patterned 
and then etched in the corresponding gate geometry in a 
plasma etch using a CyHe chemistry at about 200 W. Any 
polysilicon stringers, i.e., slender fingers of polysilicon ^ 
along the bottom edges of the gate formed during the etch on 
the substrate are removed by a post wet etch in a conven- 
tional etch, e.g., a 5-2-0.15 etch solution. 

After the first and second polysilicon deposition and 
patterning sequences, first and second gate electrode isola- 3Q 
tion oxide layers, respectively, each of about 2000 A in 
thickness, are grown over the polysilicon at about 900° C. 

The buried contact structure is defined after the second 
polysilicon deposition, patterning, and oxide growth. In this 
step, as explained previously, the location of the substrate 35 
charge-collection junction is first defined 
photolithographically, and then the silicon nitride and silicon 
dioxide layers overlaying the substrate are removed by a first 
SF 6 plasma etch and then a buffered oxide etch, respectively. 
Then the third polysilicon layer is deposited and patterned in ^ 
the manner described above to form the third MOS gate 
structures, and the FET gates. As previously explained, the 
dopant introduced during the polysilicon deposition diffuses 
through the polysilicon layer to form a p/n junction in the 
underlying substrate. 45 

In the next step, the source and drain regions of the FET 
structures are formed. Here, the location of the sources and 
drains is first defined photolithographically, and then, as in 
the previous step, the silicon nitride and silicon dioxide 
layers overlaying the substrate are removed by a first SF 6 50 
plasma etch and then a buffered oxide etch, respectively. A 
phosphorus implant is then carried out in the source and 
drain regions at about 80 keV and a dose of about 5xl0 15 / 
cm 2 . 

At this point, the electrical grounding connection sub- 55 
strate contacts are formed. The contacts are first photolitho- 
graphically defined, and then the field oxide in the area of the 
contact is etched by a buffered oxide wet etch. The contacts 
are implanted with boron at about 100 keV and a dose of 
about lxlO 14 . Then the polysilicon layers that accumulated 60 
on the substrate back-side during the deposition processes 
are etched, and an oxide layer of about 600 A in thickness 
is grown with a wet process at about 900° C. The substrates 
are then annealed in hydrogen at a temperature of about 900° 
C. for about 15 minutes to reduce surface states and traps. 65 

Metallization contacts to the CCD circuitry are then 
formed by first photolithographically defining the contacts, 
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then etching the oxide in a buffered oxide etch, and etching 
the silicon nitride and silicon dioxide off of the back side of 
the wafers with plasma, and buffered oxide etches like those 
previously described. The substrates are then cleaned, and 
aluminum is sputter-deposited to form a metal layer of about 
1.1 jum in thickness. The aluminum is then photolithographi- 
cally patterned and etched to form the metal connections to 
the CCD imager. 

At this point, the processed substrate is optionally and 
preferably annealed at about 400° C. to reduce the contact 
resistance between the aluminum and the underlying poly- 
silicon or silicon. Then a layer of silicon dioxide, doped with 
about 5-6% phosphorus, or silicon nitride, is deposited to a 
thickness of about 0.7-1.0 /*m to provide a scratch and 
ambient contamination protection layer. A final anneal in 
hydrogen or forming gas is then performed at about 400° C. 
for at least about 1 hour, followed by a slow ramp-down in 
temperature in the anneal gas. 

An optional process enhancement can be added to the 
above fabrication sequence to produce a pixel configuration 
that limits the generation of dark current charge generation 
in the buried channels of the pixels. This process enhance- 
ment lowers the dark current by suppressing the surface state 
trap current generation that can occur at the silicon -silicon 
dioxide interface at the surface of the CCD buried channel. 
This trap-based activity can be reduced by doping the 
surface with the opposite type impurity from that of the 
buried channel. The surface dopant "pins" the fermi poten- 
tial at the band edge, thereby lowering the probability of the 
capture and emission of electrons into the charge collection 
well of the buried channel. This surface doping can be 
performed at the time of the buried channel implant. 

At the end of the fabrication process, functional CCD 
pixel and register MOS structures and output circuit FETs 
exist on the front surface, i.e., the p-type epitaxial layer, of 
the starting substrates. A final processing sequence is under- 
taken at this point to produce a rugged back-side illuminated 
CCD imager. Referring to FIGS. 10A-10D, the processed 
substrate 150, including output metal interconnection 152 
and MOS and FET gates, is first coated with a layer of epoxy 
156 on the front side of the substrate, over the gates and 
interconnections, by, e.g., spinning epoxy on the substrate. 
The coated front side is then attached to a supporting silicon 
wafer, glass substrate 158, or other suitable support sub- 
strate. The choice of epoxy preferably provides a sufficiently 
hard adhesive binder that can sustain forces of ultrasonic 
wedge bonding or gold ball bonding later in the sequence. 
EPO-TEK 377 epoxy is an example epoxy that performs 
adequately. 

Then, as shown in FIG. 10B, a two-step thinning process 
is carried out to thin the substrate. Preferably, the thinning 
process minimizes pits on the etched back surface and 
provides control of etch uniformity to within about 1 /*m or 
less. In one example etch process, the substrates are etched 
in a bath 160 of HF/HNOg/CHgCOOH while being rotated 
at several rpm. Other suitable etch techniques can also be 
employed. The etch is preferably carried out until the entire 
p + -type substrate is removed and, depending on the imager 
application, some portion of the p-type epitaxial layer 
removed. The thickness of the remaining silicon substrate 
can range between a few microns and tens of microns. To 
enable high quantum efficiency of the imager in the red to 
near-infrared regions of the spectrum, where a substantial 
portion of the light in the night-time spectrum resides, a 
thinned-substrate thickness of about 20 ^m or more is 
preferred, as discussed below. 

The thinned substrate is at this point a viable back-side 
illumination surface for operating the CCD imager. 
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However, it is preferred that the back side be passivated to tion formation during high temperature processing due to 
limit dark current generation and to stabilize the spectral induced stress resulting from, e.g., temperature gradients 
response of the imager. In one example, this is accomplished across the substrate, high doping concentration of atoms 
by implanting a shallow p + accumulation layer on the back mismatched to the silicon lattice size, or differing thermal 
surface of the substrate; e.g., BF 2 can be ion-implanted at a 5 expansion coefficients between the substrate and a film 
low energy of about 5 keV to a dose of about 2xl0 14 /cm 2 . grown or deposited on the substrate. 
The metal interconnections and epoxy layer existing on the It is therefore preferable that specific precautions be taken 
composite substrate structure prohibit a conventional high- during CCD imager fabrication on a high-resistivity sub- 
temperature dopant activation anneal. Instead, irradiation strate to minimize such slip dislocations. For example, all 
with, e.g., a pulsed KrF excimer laser can be undertaken; 1Q films grown or deposited on the substrate should preferably 
such laser irradiation causes superficial melting of the be as thin as possible, taking into account their specific 
exposed back surface and results in effective activation of function, to minimize film-induced substrate stress. Thermal 
the dopant without damage to the front surface or electron- processing should also preferably be tailored to minimize 
ics. Preferably, the laser energy density is at least about 0.3 temperature-induced slip formation. For example, substrate 
J/cm 2 and most preferably, overlapping pulses, e.g., four 15 boats used to support the substrates during a processing step 
overlapping pulses, at about 0.56 J/cm 2 , are employed; this should preferably be polysilicon to lower the stress at the 
has been found experimentally to fully activate the dopant. points where the substrates are in contact with the boats. The 

At this point, as shown in FIG. 10C, a layer of aluminum substrates should be separated from each other in the boats 

164 is sputter-deposited and patterned to produce an opaque by a distance sufficient for the entire span of each substrate 

light shield over the frame storage pixel array. Then an 20 to be more evenly exposed to the thermal environment 

antireflection layer 166 is coated over the image pixel array. during the thermal processing. At the start and end of each 

One suitable antireflection coating consists of a layer of thermal process, the temperature should preferably be 

silicon monoxide, SiO. Over the wavelength range of visible slowly ramped such that the thermal gradient across the 

and near infrared wavelengths, SiO is characterized as being substrates is maintained relatively low. These various pre- 

close to the geometrical mean of air and silicon, making SiO 25 cautions provide the ability to produce a high-resistivity, 

a fairly good match to the substrate -air interface. SiO is not relatively thick CCD imager substrate that provides high 

a good choice, however, for visible and ultraviolet wave- quantum efficiency at the spectrum regions characteristic of 

lengths because at these wavelengths, SiO becomes absorb- low-light-level scenes such as the night sky. 

ing. There are a variety of other suitable antireflection layer An alternative embodiment of a back-side illuminated 

materials, e.g., beryllium oxide, aluminum oxide, or 30 CCD imager is provided in the invention to accommodate a 

hafnium oxide. wide field-of-view optical input coupling lens. A limitation 

Finally, as shown in FIG. 10D, the silicon epitaxial layer of conventional low- light-level imaging systems is their 

is etched using a standard silicon etch and the insulator 168 very limited field of view, e.g., typically about 40 degrees or 

is etched with a plasma and buffered oxide etch sequence to less, at unity magnification. The field of view of an imager 

remove those layers in the area of the aluminum metalliza- 35 is in general widened through the use of a wide field of view 

tion contacts 152 on the front surface of substrate. Then lens, e.g., a lens designed to couple input light from some 

conventional wire bonding 170 is undertaken to make elec- specified angle of view, e.g., 100 degrees. Use of such a lens 

trical connection to the CCD imager. At this point, a fully typically has several practical limitations, including the 

functional back-side illuminated CCD imager in accordance requirement of a multi-element lens system for correcting 

with the invention is achieved as shown schematically in 40 for any field distortions that are incurred by a single element 

FIG. 11, Highlighted in the figure are the three phase pixel lens. For example, as shown in FIG. 12A, input light 12 

gating structure, and fa, the implant layer 172 for coupled through a wide field of view lens 200 to the 

enabling electronic shutters, the buried channels 174, chan- low-light-level imaging system of the invention is preferably 

nel stop implants 176, and blooming drains 178 in the further focused by a correctional multi-element lens system 

channel stops. Also shown is the anti-reflection coating 166 45 202 before being directed to the back-side illuminated CCD 

covering the imaging pixel array. imager 204. The additional lens system 202 adds unwanted 

In another fabrication process in accordance with the weight to the imaging system, requires additional housing 

invention, the CCD imager is fabricated on a high-resistivity space, and adds to the overall system cost, 

silicon substrate having a resistivity of between about 1000 In addition, for a fixed pixel-resolution CCD imager, the 

Q-cm and 12,000 Q-cm and the substrate is etched back, in 50 angle of view subtended by any one pixel is increased as the 

the back-side illumination configuration, to a relatively thick field of view is widened, resulting in reduction of the angular 

layer, e.g., about 20-100 /an. Use of a thick, high-resistivity resolution of the imager. Furthermore, in cases where the 

substrate is motivated by low- light- level performance goals imaged field of view exceeds the field of view that can 

for the CCD imager, as mentioned above. A large fraction of viably be displayed on a given display apparatus, there 

the optical wavelength spectrum of the night sky is in the red 55 results a net minification of the imaged scene, i.e., magni- 

and near infrared spectrum regions. Light in these wave- fication less than unity, which is undesirable for mobility and 

length regions is characterized by a relatively long absorp- navigation tasks where judgments of spatial relations are 

tion length in silicon, meaning that a thick substrate is critical. 

preferred to achieve a high quantum efficiency at those In one embodiment of the invention, the thinned, back- 
wavelengths. High-resistivity substrates, formed by, e.g., a 60 side illuminated CCD imager is spherically curved with an 
conventional float zone process, enable a thick substrate to appropriate radius of curvature and located at a position with 
perform comparably with a relatively thinner substrate; i.e., respect to a single-element simple lens such that the image 
the modulation transfer function of the thicker substrate is plane at the CCD imager location is in focus over a wide 
not significantly degraded and the inter-pixel response char- field of view. As shown in FIG. 12B, the input light 12 is 
acteristic of the thinner substrate is maintained. 65 here directed through the simple single lens, and then 
Silicon substrates fabricated by the float zone process directed immediately to a curved CCD imager 206 in which 
characteristically lack oxygen and are prone to slip disloca- the CCD imager substrate 208 is positioned on an appro- 
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prialely curved support substrate 210. This enables imaging 
of a large field of view by the imaging system of the 
invention using only lightweight and simple optical input 
elements. The field of view dictated by the input lens 200 
need not be equal in all directions, and it may be in some 
conditions preferable to enable an extensive field of view in 
one direction, e.g., horizontal, through use of a lens having 
a component of cylindrical curvature. Correspondingly, the 
CCD imager can here also be curved to conform to the focal 
plane. If necessary for a given curvature, multiple CCD 
imagers can be mosaiced over a support substrate to cover 
the desired field of view. In this case, however, reduced 
scene resolution and image minification can occur as 
described above. 

Referring to FIG. 12C, an example process for producing 
a curved CCD imager 206 in accordance with the invention 
entails deforming a thinned CCD imager substrate 212, 
having one or frlofe individual CCD imager chips 206a, 
206b on the substrate, and securing the deformed substrate 
to a support substrate 214 having a desired radius of cur- 
vature for a given imaging application. Once the imager 
substrate 212 is secured to the underlying support substrate 
214, the composite structure can be diced to produce 
individual, curved CCD imager chips 206a, 206b. 

At the start of the process, the imager substrate is first 
thinned by, e.g., exposing the back side of the substrate to an 
appropriate etch bath while masking the periphery of the 
substrate to form a rim 213 (FIG. 12C) on the substrate for 
registration and handling during the following process steps. 

Deformation and securing of the CCD imager substrate to 
an underlying support substrate is accomplished in one 
example process shown in FIG. 12D. Here, the contoured 
support substrate 214 is positioned on a mounting plate 215 
that includes an aperture 217 connected to an exhaust system 
219. A layer of epoxy 216, as described previously, is 
deposited on the support substrate, and the CCD imager 
substrate 212 is then placed on top of the epoxy-coated 
support substrate. Alternatively, the lower surface (front 
side) of the CCD imager substrate can be coated with epoxy. 

A flexible plastic sheet 218 is then placed over the upper 
surface of the CCD imager substrate, and the exhaust system 
is activated to evacuate the air from beneath the plastic 
sheet. This produces a pressure 220, of up to, e.g., about 1 
atm, that is applied very uniformly over the entire CCD 
imager substrate, gently forcing the substrate into exact 
conformity with the underlying contoured support substrate. 
The pressure condition is maintained preferably until the 
epoxy has cured, after which the vacuum is relieved. At this 
point, the composite substrate assembly is diced to produce 
individual CCD imager chips. 

Other curvature-forming processes are contemplated by 
the invention and the foregoing is but an example of one 
suitable process. This technique is suitable for both back- 
side illuminated and front-side illuminated CCD imagers; in 
the case of a conventional front-side imager, the diameter of 
the contoured support substrate is preferably not greater than 
the inside diameter of any support rims on the periphery of 
the substrate to permit bonding of the back-side of the 
substrate. As will be recognized by those skilled in the art, 
the curvature -forming process is applicable to any focal 
surface profile; i.e., the thinned substrate can be deformed to 
provide multiple radii in any reasonable combination to 
accommodate a specific lens. The invention is therefore not 
limited to a specific curvature or profile; the concave profile 
described above is provided only as an example of the many 
profiles possible. 
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In a further alternative CCD imager embodiment in 
accordance with the invention, a four-phase MOS pixel 
configuration is provided to enable two dimensional charge 
packet transfer through the CCD imager. Typically, CCD 

5 imager configurations support only one-dimensional transfer 
of charge packets through the imager; for example, in the 
back-side illuminated imager described above, charge pack- 
ets are transferred from an imaging pixel array to a frame 
storage pixel array in regimented columns, and are trans- 

10 ferred out of serial registers out of a single register stage row. 
Charge packets cannot in this configuration be moved 
between columns in the pixel arrays. In the alternative CCD 
imager embodiment, charge packets can be transferred both 
from one pixel row to another along a given pixel column 

1S and from one pixel column to another along a given pixel 
row. This enables true two dimensional transfer of CCD 
charge packets, providing the ability to shift charge packets 
in the CCD imaging pixel array to track a moving image 
during the image integration time, thereby eliminating 

20 so-called blurring of the acquired image. 

Referring to FIG. 13, a pixel 230 of the two dimensional, 
or orthogonal-transfer CCD provided by the invention con- 
sists of four MOS gate structures per pixel, 4>i, <t>2> <t>3, and 
<j> 4 . Preferably, the fa gate is biased low to provide a channel 

25 stop mechanism 232 and the <j> 2 -^4 gates are clocked in 
conventional three-phase manner to transfer charge 
vertically, i.e., along columns. Likewise, the fa gate can be 
biased low and the $ ly 4> 3 , and fa gates can be clocked to 
transfer charge horizontally, e.g., along rows. Conventional 

30 channel stop implantations 234 are required under the inter- 
section of the ((>! and fa gates. Other than these 
modifications, the orthogonal-transfer CCD imager can be 
fabricated using the process sequence given above, with the 
fourth gate being provided either by the aluminum intercon- 

35 nect or more preferably, with an additional polysilicon layer. 
With this control scheme, charge packets accumulating in 
the CCD image pixel array can be manipulated to "follow" 
a moving image during the image integration time. This 
ability is particularly advantageous for ground-based low- 

40 fight-level astronomical imaging. Generally, atmospheric 
turbulence produces random phase distortions in the optical 
wave front of an astronomical image. This tends to severely 
degrade the image resolution to well below the effraction 
limit of conventional large telescopes. Much of this distor- 

45 tion manifests itself as a tilt in the wavefront, and thin in turn 
leads to a translation of the image at the imaging focal plane. 
Conventional adaptive-optics systems compensate for this 
lowest-order distortion by using a fast tip-tilt mirror in the 
optical path. The orthogonal-transfer CCD imager provided 

50 by the invention can, however, correct the image electroni- 
cally by shifting the charge packets in the image pixel array 
during the integration time to maintain registration with the 
moving image. Mechanical resonances are typically a chal- 
lenge for tip-tilt mirrors, but even a large orthogonal-transfer 

55 CCD imager with resistive polysilicon gates can easily 
follow motion, e.g., even at a rate of about 1 kHz. As with 
conventional systems, the image movement and control 
information must be independently measured and processed, 
e.g., with a high-frame-rate imager viewing a bright, nearby 

60 star, or possibly using a fraction of the light from the object 
of interest. Other features and advantages of the orthogonal- 
transfer CCD configuration are described by Savoye and 
Burke in U.S. patent application Ser. No. 08/563,983, filed 
Nov. 29, 1995, the entirety of which is hereby incorporated 

65 by reference. 

Turning now to other components of the low-light-level 
imaging system of the invention, the system's CCD imager 
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current generated by a CCD imager can be reduced to about 
0.05 to 0.1 nA/cnr at about -5° C; this is a dark noise 



preferably is provided with means for suppressing any dark 
current generated in the imager by parasitic effects not 
related to a scene being imaged. Such dark current, which is reduction by a factor of 5-10 with respect to room- 
primarily thermally generated, is generally somewhat non- temperature operation. This reduced dark current makes less 
uniform from pixel to pixel in a CCD imager, and therefore 5 critical the background image noise subtraction that con- 
can add a spatially fixed noise pattern, as well as temporal ventionally must be carried out before viable imaging can 
shot noise, to the image signal produced by the CCD imager. be g m m a CCD HDa S cr - 

This is particularly important for imaging low-light-level As mentioned above, other imaging system components, 

scenes in which the acquired image may be of such low the sl S nal processor 24, can be included in the 

intensity that uncontrolled dark current could swamp the 10 CCD imager package. Turning to FIG. 15, the analog signal 

acquired signal. Even aside from low-light-level imaging, processor performs amplification as well as noise reduction 

however, low overall noise operation of the imaging system functions on the CCD imager output signal 160. One analog 

of the invention is achieved by reduction of the CCD imager si S nal Processor is provided for each CCD output; thus for 

dark current noise contribution. a four-output CCD four analog processors are employed. 

... , -j , , , . r ,c Each analog signal processor can preferably include an 

In one embodiment provided by the mvention for sup- 15 4 , , ■ i_ .l * j • ** p 

™^ . j , / ^ adjustable gam setting such that during operation of the 

pressmg CCD imager dark current, a cooler, e.g., a thermo- . . . , & u .% K. , . , frt 

i . • 1 /^7>\ • 1 j\ i *L %r-r* • imaging system, balance can be periodically adjusted to 

electric cooler (TEC), is employed to cool the CCD imager . 6 . P * . w . , , v ... J . . J pm 

, .. . , . v , " « • • ♦ 0 6 mamtain gain and offset balance with respect to all CCD 

and limit the dark curren generated in the imager. Because recognized by those skilled in the art, 

dark current is exponeotiaUy dependent on temperature, me ^ are a ^ de f techniques> both manual 

modest coohne, which can be effectively provided with a LKi - . .° , . , , , • • _ 

, . t j *i p r*t_ 4-1-/-, ■ • and automatic, that can be employed to mamtain this pre- 

TEC mecnamsm, can greatly reduce it. Jne J hC mechanism f , ... , . . U1 

r ui i j * j • * . + %u *u * ferred multiple-output balance, 

is preferably placed in direct contact with the CCD imager ^ , . , . ^ . _ 

in a configuration that renders the CCD temperature stable E "? ™^ el ° ut P u ' ?*»»! » filst P?** 88 * ^. 3 b,1 . ffer - 

and controllable but that does not introduce turbulence to amplifier250, which dnves the capacU.ve load of me winng 

input light incident on the CCD. « «•» following preamplifier stage 255, and increases the 

. image signal level above that of the characteristic noise level 

Referrmg to FIG. 14, one example CCD imager package of ^ foUowing ^ages. ^ 5uffer amplifier is preferably in 

235 that accommodates a TEC mechanism is configured a Mim3e _ foUower configuration, and can consist of, e.g., a 

with a TEC 236 on top of which is mounted the CCD imager mQ9 pK:hannel jp EX with a 2 kQ active load. The input 

204, shown here as a planar, back-side illummated imager. referrcd noise of the buffer ^ bclow that of the CCD imager 

TTie TEC consists of any suitable thermo-electric device, output ^ faas a bandwidth of about 100 MHz when 

e.g., a thermoelectnc cooler from Melcor, of the FC series. drivi a 10 pf load p^fc^ t he buffer is housed on a 

TOs TEC is characterized by an unloaded temperature Qybrid substrate located mside the CCD package> M 

differential of about 65 C. described earlier. 

Power and control from the imaging system power source 3S Following the buffer is a single-pole low pass filter 253 

18 and temperature controller 20 are provided to the TEC ^ a 3 dB bandwidth of about several megahertz. The low 

through the package leads 238. The CCD imager 204 is pass filter bandwidth lirmts the high frequency thermal noise 

positioned under a package window 240 such that light input that or igm at es at the CCD output circuit and the buffer stage 

to the imaging system can be directed straight to the CCD and ^ passe d to the preamplifier stage 255. The preamplifier 

image pixel array. Connected to the CCD imager 204 are one 4Q 2 55 provides a gain of about six to ten and consists of, e.g., 

or more of the signal processing electronics provided by the ^ AD 82 9 op-amps. The first op-amp performs impedance 

imaging system. For example, the analog signal processor transformation and provides most of the gain. The second 

24 (FIG. 1) and A/D converter 26 can be included in the op .amp has modest gain and provides signal inversion and 

cooled package. matching to the following correlated double sampler stage 

In the CCD mounting configuration shown, only the CCD 45 260. The AD 829 has a very wide dynamic range that is large 

imager 204 is directly cooled, resulting in minimization of enough to accommodate the dynamic range produced by the 

power requirements to effectively cool the structure. The CCD imager of the invention and the AC coupled drift. The 

package substrate 242, which acts as the heat sink for the input referred noise of the AD 829 is low and the settling 

TEC, is heated a few degrees higher than the surrounding time on the order of a few nanoseconds, making them a good 

ambient, advantageously preventing moisture condensation 50 choice for real time video imaging applications, 

on the package. During nominal room temperature The preamplifier stage is followed by a second single-pole 

operation, the TEC can maintain a CCD temperature of i ow pass fitter 257. The amplified and filtered signal is 

about -5° C. and maintain a package temperature of about passed to a correlated double sampler (CDS) stage 260, 

25° C. which removes reset and low frequency noise due to the 

A thermistor temperature sensor, preferably located in the 55 CCD imager source-follower output circuit, using a FET 

package adjacent to the CCD imager, can be employed to circuit 262 and temperature-insensitive voltage reference 

monitor the CCD and package temperature and deliver 264. This stage also eliminates any slow drift component, 

temperature readings to the temperature controller 20. One either thermal or ac-couple related, of the amplifier 255. The 

suitable temperature controller is the New England Photo- CDS also establishes a black level reference for absolute 

conductor Controller, Model TCI, which can maintain the 60 zero. The CDS consists of, e.g., an SD 210 clamp MOSFET 

temperature to within a fraction of a centigrade degree. For transistor followed by an AD9100 track-and-hold 266. The 

a modest size imager operated at frequencies of a few track-and-hold 266 is linear over a range of about -1V-+1V, 

megahertz, the power dissipation of the TEC is about several and accommodates at least about 11-12 bits of resolution. A 

watts. The temperature controller adjusts the TEC control 75 Q buffer (not shown) receives the signal from the 

signal using, e.g., a conventional voltage -controlled feed- 65 track-and-hold. This buffer can transmit the signal with 

back loop that maintains temperature stability. With such a minimal degradation over a 75 Q cable of about 25 feet or 

control scheme, it was experimentally found that the dark shorter. 



01/14/2004, EAST Version: 1.4.1 



5,909,244 

33 34 

The buffered signal is then passed to the A/D converter FIG. 16B is a state machine diagram illustrating the 

and formater 26 (FIG. 1) for digitization and for formatting sequence of states controlled by the READ and RAB signals 

the pixels into a sequence of configured image frames of for an example 128x128 CCD imager employed in the 

pixels. The dynamic range of the pixel values presented to imaging system of the invention. Beginning with the state at 

the A/D converter is preferably larger than 8 bits and can be 5 & G t0 P of the diagram, if the RAB signal is low, the CCD 

12 bits, or larger, e.g., 16 bits, depending on the application. ima S er enters the frame integration mode, stare, and remains 

Thus, an appropriate digital bit-sized converter is selected at that state untl1 a READ P??* 15 received. Receipt of the 

for a given application. Suitable A/D converters include the P ulses causes a tra u nsitl0n to th f ^™ transfe ' state ' 

12-birComlinear CLC 925 hybrid A/D circuit. The 12-bit ™ e m *& T continues in this state until all charge packets in 

, CtL . / t , c • 1 ii *i_ * „ the imagine pixel array have been transferred to the frame 

d ^ am * r£m S eo thisconverterm atcbes: fairly well with tot 10 * > f ^ ^ ^ of ±c k 128xl2g 

needed for digital image processing as discussed below. The ^ * m ^ ^ M tQ letel transfer the 

Analog Device AD9220 monolithic A/D converter, A/D pixel charge packet array. 

converters from Burr Brown and other Comlinear converters ^ imagef thefl entefS ^ chafge dump ^ remaining 

are also suitable. ^ that sUte umil the output re gister has been cleared. Upon 

Referring back to FIG. 1, the timing controller 22 pro- 15 completion of the register clearing, the row of pixels values 

vides overall timing control of the A/D, the analog signal is output as electronic pixel signals to the analog signal 

processor, and the CCD imager to synchronize image acqui- processor. This is continued until all rows of charge packets 

sition and signal processing. One example timing controller of the imaging pixel array have been output. At the end of 

in accordance with the invention consists of a programmable the output, the timing controller awaits control from the user 

logic device that embodies a control state machine for 20 by way of the RAB and READ lines to determine the next 

synchronizing image integration, image signal readout, ana- state, either RAB or STARE. The imaging cycle is then 

log processing, A/D conversion, image frame sequence begun once again. 

formatting, and digital image processing. FIG. 16C is an example timing diagram for controlling the 

1T 1 etimingcontroller22operatesbasedoncontrolsignals ™»f8 s y stem * ™ e first 9 si f ^ a three-phase 

•j j u *u *i u- u ■ i ~* 25 clocking operation for an example three-phase CCD imager, 

provided by the user controller 32, which can consist of a ^ jj^ (LS) Frame S / QC (FS)j jfo Encode ^ 

personal computer, customized hardware or other suitable and {xc{ ^ (p ' CLK) s{ ^ m Med tQ ^ ^ 

configuration. The user controller preferably includes input/ coaverler and formater for configuring the pixels, once they 

output capabilities, e.g., a display and keyboard or control afe digitized> mto a sequence of image frames. The formater 

switches, for enabling user selection of imaging system ^ men passes the LS, FS, and PCLK signals to the digital 

operating parameters by, e.g., a pull-down menu configura- processor 28 (FIG. 1) for synchronization. The digital 

tion or other control switching configuration for enabling processor runs asynchronously with respect to the 

user selection. In one user control configuration, the user imager, but preferably maintains real time throughput of the 

controller 32 monitors the activity of all processors and the pixel data a i aten cy time of no more than about 

A/D and formater and is the controlling mechanism for 0Qe f rame tm3e 

changing the status of the processors and A/D and formater. 35 F , G fc aQ example timing diagram for formatting the 

In one example timing control configuration, the user can digitized pixels into a sequence of image frames. The 

select between three operational modes, namely free run formater, consisting of a general processor, customized 

mode, frame control mode, and full control mode. In free run hardware, or other configuration, applies the timing control 

mode the CCD imager runs at its maximum operable speed; 4Q to the sequence of digitized pixel values produced by the 

in frame control mode, the CCD imaging pixel array inte- a/D. In the case of an example CCD imager having four 

gration time is user-controlled; in full control mode, both the output ports, there are four corresponding sets of analog 

integration as well as extinction of charge packets acquired s j gna i processors and A/Ds; at the formater, the four streams 

in the CCD imaging pixel array is controllable. 0 f digital pixel values are multiplexed down to one pixel 

FIG. 16 A is a timing diagram for each of the three 45 stream running four times faster than the individual streams, 

example user-selectable modes. In all three cases, the user This multiplexing is preferably performed in the digital 

controller produces two pulses, READ and RAB signals, for domain to maintain low noise in the signals. In the diagram, 

controlling the CCD imager. In all three diagrams, each D0-D11 are the 12 digital pixel bit lines output from the 

READ pulse width is preferably about 2 jsec and the RAB formater, and the four parallel CCD imager output port 

pulse width is preferably at least about 3 jHsec. In free run 50 clocking phases are illustrated. 

mode, both the RAB and READ signals are maintained low. The digital output of the A/D converter and formater, as 
This causes the CCD imager to acquire charge packets and explained above, is a sequence of digital pixel frames 
output corresponding signals at the maximum operable representative of me image acquired by the CCD imager 16. 
frame rate; the Frame Sync signal therefore sets the frame As a consequence of the extreme light sensitivity and low 
rate. 55 noise characteristics of the CCD imager, a very large 
Frame control mode is enabled by pulsing the READ dynamic range of digital imagery is supported by the low- 
signal. In this mode, the integration time for a given frame light-level imaging system of the invention. Indeed, digital 
is set by the time that elapses between READ pulses, as images of low-light-level scenes at video frame rates with 
shown in the diagram; the READ pulse triggers the Frame image pixels of, e.g., 4096 useful gray levels or more are 
Sync pulse. Full control mode is enabled by pulsing both the 60 supported by the system; an output bit stream of, e.g., 12-bits 
READ and RAB signals. In this case, if the RAB pulse is or more, is provided to represent the full scale of gray levels, 
high, the CCD imager clock phases run backward, thereby This very wide dynamic range has many advantages for 
extinguishing a charge packet signal out of a drain. The time observing a low-light-level scene such as a night scene, that 
elapsed between the falling edge of the RAB signal pulse has both bright and dark regions in it, or a scene of low 
and the rising edge of the READ signal pulse sets the 65 contrast, e.g., caused by fog or haze, 
integration time for a given frame, with the rising edge of the The wide dynamic range presents a challenge, however, 
READ signal causing a frame to be output. for display of the imagery on a grayscale or color monitor or 
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other display, like those discussed earlier, because most 
displays are designed for display of only 256 levels or less 
of monochrome digital imagery on 8 digital bits or less. 8-bit 
digital imagery is often preferable because it conforms to the 
general sensitivity of the human eye. With this display 5 
constraint, it is necessary to compress the dynamic range of 
the imaging system to support a given display range capa- 
bility. Importantly, this compression is preferably achieved 
while at the same time preserving the important image data 
acquired from a scene. Moreover, this dynamic range com- 1Q 
pression is preferably achieved in real time, i.e., at video 
rates with minimal latency, e.g., one frame delay or less, to 
support the real time video capability of the other imaging 
system components. 

In the invention, the digital image processor (28 in FIG. 5 
1) provides low-light-level adaptive dynamic range 
compression, enabling both local contrast enhancement and 
local automatic gain control within the acquired image. 
Local here refers to individual pixels in an image and 
adaptive here refers to each pixel in the context of its 2Q 
neighboring pixels in the surrounding image area. This 
localized adaptive processing generates, at video rates, out- 
put images in which useful contrast information in both the 
light and dark regions of the image is preserved, and 
conforms the digital signal to the dynamic range constraints 2$ 
of a selected display. 

Referring to FIG. 17, the digital image processor 28 
accepts from the A/D converter and image frame formater a 
digital image signal 300, which comprises, e.g., a 12-bit 
digital signal. Four sub processors provide the digital pro- 30 
cessing capability, namely, a center-surround shunt neural 
network processor 302, an image statistics processor 304, a 
look-up table processor 306, and a remap processor 308. 

The center-surround shunt processing performed by the 
shunt processor 302 is characterized by a central excitatory 35 
region, preferably a single pixel, and a surrounding inhibi- 
tory region, preferably a 5x5 pixel neighborhood, or other 
neighborhood size, e.g., a 9x9 neighborhood created by a 
separable 9x1 convolution. This type of computation is 
suggestive of contrast processing performed at the retinal 40 
stage of the human visual system, and is described in detail 
by, e.g., Ellias and Grossberg, in "Pattern Formation, Con- 
trast Control, and Oscillations in the Short Term Memory of 
Shunting On-Center Off-Surround Networks, Biological 
Cybernetics, 20, 69-98, 1975. 45 

The shunt processor 302 accepts input imagery of arbi- 
trary dynamic range, enhances the contrast of the image 
based on data local to each pixel in the image, adaptively 
normalizes the enhanced result for each pixel, based on data 
local to each pixel, and then guarantees the resulting image 50 
to lie in a prescribed dynamic range, preferably, e.g., in the 
range of -1 to +1. 

The resulting image is then globally rescaled on a pixel- 
by-pixel basis by way of a look up table (LUT), constructed 
by the LUT processor 306, to the dynamic range suitable for 55 
a given display, e.g., a range of 256 gray levels. The remap 
processor 308 provides the actual reseating of the shunted 
image dynamic range. This remap processor can adapt to 
temporal changes in the dynamic range of the input image 
scene as a result of the real time functionality of the LUT 60 
construction processor stage 306; the remapping table can 
thereby be recomputed for nearly every digital video frame, 
based on the shunted image statistics gathered from earlier 
frames by the image statistics processor 304. This enables 
temporally-adaptive nonlinear remapping of the shunted 65 
output dynamic range to the fixed dynamic range of the 
selected display hardware. 



36 

Considering the first processing stage, the center-surround 
shunt processor 302, this shunt processing functionality can 
be implemented in a variety of suitable embodiments, as will 
be understood by those skilled in the art. Preferably, the 
neurodynamics and equilibrium of the so-called center- 
surround shunt receptive field is described at each pixel 
location by the relations: 

dE u r (1) 

= -AEtj + (1 - Ei,)[Cf% - (1 + E S M, * /*]..; 



where 



or 



ij ~ A + [C1 C + 0,*/%- * 

where ij are the row and column coordinates of a given pixel, 
E is the opponent processed enhanced image, I c is the digital 
input image, from the A/D converter, that excites the single 
pixel center of the receptive field, and I 5 is the digital input 
image, also from the A/D converter, that inhibits the sur- 
round G s of the receptive field, which is convolved against 
I s . 

The surround is modeled as a Gaussian weighted local 
neighborhood. Referring to FIG. 18 A, a preferable 5x5 pixel 
neighborhood of weights is provided; in FIG. 18B, a pref- 
erable one -dimensional 9x1 pixel neighborhood of weights 
is provided, for enabling an efficient implementation of a 
two-dimensional 9x9 pixel neighborhood by means of a 
separable convolution using only the one-dimensional 
weight neighborhood. 

The first relation in equation 1 above models the temporal 
dynamics of a charging neural membrane leaking charge at 
a rate A, and has excitatory and inhibitory input ion currents 
determined by Ohm's law; with the shunting coefficients 
1+E acting as potential differences across the membrane, 
and the input image signals modulating the ion-selective 
membrane conductances. The second relation in equation 1 
above describes the equilibrium that is rapidly established at 
each pixel, i.e., at the frame rate, and defines a type of 
nonlinear image processing with parameters A, C, and the 
size of the Gaussian surround. The third relation above is an 
alternative expression of this equilibrium, explicitly 
accounting for the spatial contrast term. Preferably, C is set 
equal to 2, and A is set equal to 10<I>, where <I> is a 
measure of the mean brightness of the input digital image or 
that of a previous image, although other values of C and A 
are also suitable. Indeed, the exact choice of the parameter 
A can be selected by the user at any moment of the imaging 
system processing, input by the user controller (32 in FIG. 

!)• 

As stated previously, the input to the first processing stage 
302 is typically 12 bits or more, derived from the analog 
CCD imager output signal. The output of the first processing 
stage should be computed to an integer precision of at least 
10 bits, and preferably 11 bits. The precision should in any 
case exceed that of the intended display hardware. With the 
output of the second relation of equation 1 known to be 
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bounded such that -1<E<1, the result of the computation is The remap processor 308 applies the LUT remapping 

transformed to the desired interval corresponding to 11 bits function to each pixel of the image processed by the shunt 

via an additional computation such as 1024(E+1). This processor 302. Thus, each output value from the shunt 

insures that the output will be in a form suitable for the processor, which preferably is an integer in the range of 

remap processor 308 . 5 0-2047, serves as an address into the LUT The output of the 

Preferably, the third relation in equation 1 is employed LUT for a given a ddress is then an 8-bit number in the range 

rather than the second relation. In the third relation, the of q_255, meant to drive an intended display. In order to 

second term of the numerator represents the local spatial maintain rea] time flow of i^ge data through the processor 

contrast. This spatial contrast can be either positive or ^ mmimum latency the LUT applied 5y the remap 

negative, or zero. While the inclusion of this term acts to tn _ . f , , _ , . , J\ • . ^ 

f \ - , , j « . > j. » • • i 10 processor is preferably constructed previous to the time 

enhance the contrast of the output image, it is often desirable , , _ t . .J, « . . f ,. .„„, OT XM 

to only enhance positive contrast, i.e., local brightening, and * hen the P lxel dala 15 , T ^ P h 

not to enhance negative contrast, i.e., local darkening, in an ™ e r f ma P Processor 308 thereby produces a remapped 

image. This can be appreciated particularly in images of dl S 1 | al 0Ut P ut n0 c ^ { 1S delivered to e.g. dynamic 

night scenes in which stars against a dark sky are locally random "f^ memorv > for stora S e > or directl y t0 the 

brightened by their positive contrast with respect to the sky, 15 driver 31 ( FIG - X )> e -g- r a v J deo - random acce ss memory, for 

but dark rings will be formed around stars due to the conversion to analog signals which drive a display monitor 

negative contrast of the sky with respect to a nearby star in ^3 (FIG. 1) in real time. 

this enhancement mode. In order to remove such visually The adaptive dynamic range compression provided by the 

distracting artifacts, it is preferred to enhance only positive digital processor can be iterated to provide further enhanced 

spatial contrast. Thus, when computing the numerator in the 20 contrast of an image. That is, the resulting processed imag- 

third relationship, the second term is reset to zero if it would erv can be reprocessed by the same algorithm. Due to the 

otherwise be negative. The resulting output dynamic range nonlinear nature of the algorithm, two iterations cannot be 

is then bounded according to 0<E<1, and this is transformed ^P^ related to a iteration, however. In some 

to the desired 11 bit interval via the factor 2047E. implementations of the low-light-level imaging system of 

The output of the center-surround shunt processor 302 25 ^ c invention, it may be preferable to iterate the digital 

feeds both the statistics processor 304 and the remap pro- processing multiple times to enhance certain subtle contrasts 

cessor 308. The image statistics processor 302 gathers m original imagery. 

several statistics from this output, as well from the input An imaging system in accordance with the invention was 
imagery 300. In particular, the statistics processor 304 configured with the dynamic range compression function- 
computes the mean, /«, and standard deviation, a, of the 30 alitv of the digital si S nal Processor to experimentally 
output shunted imagery; or alternatively, computes two observe the results of compression functionality. The 
other parameters that capture the same qualitative informa- imaging system was configured with a back side 
tion. This processor also computes the value of the param- Ruminated, frame transfer CCD imager having 128x128 
eter A to be used in computation by the shunt processor 302 P«d*. and four 0Ut P ut P° rts each °P erated at about 2 75 
for the next following frame. Parameter A is selected to be 35 MHz ^ a TCC was employed to reduce the CCD imager dark 
proportional to the mean <I> of the input image brightness, current - ^ aild formater of the imaging system were 
as reflected in the data-dependent term in the denominator of configured to produce a digital video image sequence having 
the second and third relations of equation 1, and preferably a dynamic range of 12 bits and a frame rate of about 30 
is selected as A-10<I> frames per second. The imaging system was oriented to 
The statistical parameters generated by the statistics pro- 40 various scerjes ^der conditions, correspond- 
cessor 304 are fed to the LUT construction processor 306 in in § to a illuminance of about 1.86 mLUX. 
order to construct a LUT that will be used by the remap Referring to FIGS. 19A and 19B, three different scenes 
processor 308 to transform the value of each pixel output were presented for imaging; the first being a resolution chart, 
from the shunt processor 302. The LUT consists of a set of *e ^cond being the resolution chart with a human face in 
values equal to the number of possible input values derived 45 front of ^ chart > and me thlrd bem § me resolution chart 
from the shunt processor. Preferably, the shunt processor ™ ih a human face and a emitting diode (LED) in front 
computation is performed with an integer precision of 11 of the chart First > cach of thc thrce was una S ed t0 
bits; and in this case, the LUT contains 2048 values. In produce a corresponding 12 bit digital video image output 
general, the LUT should be preferably constructed to admit after the s y stem and ^rmater, i.e., output 
input values of 10 bits or more, corresponding to 1024 or 50 dynamic range compression applied to the image, 
more table entries. The LUT values constitute a nonlinear and then for each scene > a dynamic range of only 8 bits out 
mapping so as to remap the pixel values provided by the of the produced 12 bit image was selected for display; in 
shunt processor to the dynamic range of the intended display each case the 8 bit dynamic range particularly chosen with 
hardware, e.g., 0-255 gray scale levels or less, depending on «*P«* to the od & nal 12 bil dvnamic raD § e was based on the 
the quality of the intended display hardware. The LUT 55 P^ticular intra-scene light levels of the given scene. Then 
values are computed by the LUT construction processor 306 me same "«* 12 " blt ima Sf were processed with the 
according to a relation that provides qualitative results dynamic range compression functionality in place. In this 
resembling the following sigmoid: case > the digital image processor compressed the 12 bit 

digital video sequence output by the A/D and formater to 

£^127.5+127.5^11/20(^-/0], (2) 6Q prot j uce a corresponding 8 bit digital video sequence, 

where tanh corresponds to the hyperbolic tangent function. FIG. 19 A is a photograph of each of the three imaged 

For each possible input value which is between 0-2047 scenes produced without dynamic range compression pro- 

preferably, equation 2 yields a corresponding output value cessing; FIG. 19B is a photograph of the same three images 

in the range of 0-255, computed to 8-bit integer processed with the dynamic range compression. In all three 

precision. Thus, the LUT provided by the LUT construction 65 cases, the processed images provide a higher degree of 

processor 306 is, in the preferred implementation, an 11 contrast and therefore better define the various features in 

bit-in, 8 bit -out remapping function. the image. This is most dramatically illustrated in the last 
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image in which an LED is included; the adaptive processing division of numerator by denominator directly. In either 

capabilities of the digital image processor enable viable case, the resulting shunt computation should provide an 

imaging of the contrast defining the man's face while also output with at least 10 bits of integer precision, and prefer- 

imaging the bright LED close to the face as well as the ably 11 bits. The same integer processor can keep track of 

resolution chart in the background. These examples clearly 5 the required statistics, construct the remap LUT of 2048 

illustrate the powerful imaging capability enabled by the values ^ store it m orjHCn j p SRAM, and apply the remap 

dynamic range compression functionality of the invention. LUT to the shunt output imagery. For larger image frames, 

Tliis functionality enables display of objects in a scene 0 r faster frame rates, multiple mteger processors can be used 

across the entire original dynamic range of the scene while m ^ Qn sub _ im In the case of tne TMS320C80, 

enhancing contrast in the scene such that features not rt *, u ■ « u- u- u u 

,i , t_ T j*. *i_t_i 10 there are actually four integer processors on-chip which each 

normally resolvable are distinguishable. r '« - . r . , „ 

r> - . u ? ♦ r .u j- *• i • process one-iourtn or the image pixels, as well as a master 

Considering various embodiments of the digital image v , . , TT ™ 

p„_ — , a ' a ,v> r a ° processor which can be used to construct the remap LUT. 

processor tor providing the dynamic range compression r . r . . , , • j m_ j 

functionality, the various processing stages of the digital In lhe case of the s**™* hardware option described 

image processor form a computational pipeline. If T is the above > ie > use of an 1Dte § er processor such as the 

number of the image frame processed by the shunt 15 TMS320C80 supplemented with dedicated LUT hardware, 

processor, then the image statistics gathered by the statistics mc integer processor is responsible for computing the 

processor correspond to frame T. Once the statistics proces- numerator and denominator of the second or third relations 

sor has gathered its data from frame T, the LUT construction in equation 1, preferably to 10 bits and 6 bits of accuracy, 

processor constructs a LUT to be used on a later frame, e.g., respectively, as well as keeping track of image statistics and 

frame number T+l or T+2. Meanwhile, the remap processor 20 computing the remap LUT. The resulting numerator and 

308 applies to frame T, as it is output from the shunt denominator are concatenated to form a 16 bit word that is 

processor, a LUT constructed during frame T-l using sta- output from the integer processor directly to a dedicated 

tistics derived from frame T-2. It is also possible to use a "divide" LUT, preferably of 64K words, K«1024. This 

LUT based on statistics from frame T-l by including a delay "divide" LUT treats the 16 bit input word as an address into 

buffer between the shunt processor and the remap processor, 25 its 64K entries, and outputs a result corresponding to a 

as long as the delay is of sufficient length to hold the output preprogrammed "divide operation" that is preferably at least 

pixel values from the shunt processor while the LUT is 10 bits of integer precision, and more preferably 11 or 12 

constructed. In general, this buffering is not necessary bits long. This output is recirculated to the integer processor 

because the remapping LUT does not change much from one in order to gather shunt image statistics, and also forms the 

frame to the next 30 input to the dedicated "remap" LUT. This "remap" LUT is 

There are many suitable hardware implementations of the not preprogrammed, but rather is periodically rewritten by 

digital image processor computational pipeline, including the integer processor, which recomputes the "remap" LUT 

programmable integer processors (PIPs) that support 16 bit based on the most recent shunt image statistics. The dedi- 

arithmetic working in conjunction with random access cated "remap" LUT then outputs the final 8 bit integer pixel 

memory (RAM) for image storage, PIPs working in con- 35 value to storage or to the display drivers, 

junction with dedicated LUT hardware, e.g., RAM, for the In the case of entirely dedicated hardware, all required 

LUT, as well as RAM for image storage, and entirely multiply and divide operations can be carried out by, e.g., 

dedicated hardware, e.g., custom processors consisting of preprogrammed LUTs, none of which exceed 16 bits input 

LUTs and adders realized by, e.g., a set of field program- with 16 bits output, plus a "remap" LUT as described above, 

mable gate arrays (FPGAs) or application specific integrated 40 All data flow, additions, subtractions, and shift operations 

circuits (ASICs) with RAM, working in conjunction with can be achieved through the use of FPGAs or ASICs. Stored 

additional RAM for image storage. 12 bit image pixels from DRAM can be pipelined through 

From the standpoint of flexibility, a system built around the dedicated hardware, and output as 8 bit processed pixels 

PIPs is preferred. From the standpoint of portability of the back to DRAM for storage or directly to the display driver, 

system, including considerations of size, power, weight, etc., 45 As will be recognized by those skilled in the art, other 

the dedicated hardware configuration is preferable. hardware configurations can be employed to embody the 

In the case of a PIP and RAM system, a powerful integer digital image processor of the invention. Furthermore, the 

processor such as the recently available Texas Instruments adaptive and real time dynamic range compression provided 

TMS320C80, running at 50 MHz or faster can process 12 bit by the digital image processor can be applied to a wide range 

low-fight CCD imagery with TV resolution (roughly 640x 50 of imaging applications beyond the low-light -level imaging 

480 pixels) at video rates. By bringing image data stored enabled by the imaging system of the invention. For 

off-chip in dynamic random access memory (DRAM) into example, the dynamic range compression is applicable to 

one of the processor's static random access memory digital imaging such as X-ray, magnetic resonance imaging, 

(SRAM) banks, the processor can compute the numerator tomography, and other imaging applications in which a wide 

and denominate r of the second or third relations of equation 55 dynamic range of rich image information may need to be 

1, and can approximate the required divide operation in compressed to accommodate display hardware of a lower 

either of two ways; via a "shift right" of the numerator based dynamic range; with use of the digital image processor, such 

on at least the two most significant bits (MSBs) of the information can be compressed for display in real time as it 

denominator, the second MSB determining the "rounding is produced. Forward-looking-infrared imagers (FLIRs), 

up" to the next higher power of 2, or in the preferred 60 e.g., scanning type FLIRs, focal plane arrays, and other 

implementation, by storing in on-chip SRAM a look-up similar imagers also can benefit from the dynamic range 

table that accommodates at least 9 bits in and 16 bits out, compression functionality. In a given application, the 

which provides the inverse of an input such as the computed dynamic range compression functionality is preferably run 

denominator and then multiplies this against the computed at a rate commensurate with the digital data rate provided by 

numerator to form the required quotient. 65 the imager and system components which precede it. 

Alternatively, a processor with a sufficiently fast integer There is no specific field of image processing applications 

or floating point divide operation can carry out the required to which the dynamic range compression functionality of the 
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invention is limited. Indeed, any two- or more dimensional Referring to FIG. 20 there is shown a plot of the quantum 
digital imagery of a dynamic range that is wider than an efficiency, in percent, as a function of the wavelength of light 
available display or even follow-up processing hardware can input to an imaging system in accordance with the invention 
be processed by the digital image processor of the invention and one conventional vacuum tube-based intensifier system , 
to produce a real time, adaptive compression of the imagery 5 known as the Gen III system, i.e., the third generation U.S. 
to a desired dynamic range. In general, all that is required of military technology for image intensifier tubes, which 
the digital imagery is that its digital representation be employ a gallium arsenide photocathode. Quantum effi- 
characterized by signal integrity that supports the needed ciency is here defined for the imagers as the ratio of the 
number of bits, e.g., greater than 8 bits, to fully represent the number of photoelectrons produced in an imager to the 
imagery as that imagery is input to the digital image pro- 10 number of incident photons, for a given photon wavelength, 
cessor. Thus, the digital image processor and its adaptive, The data was obtained using an imaging system configured 
real time dynamic range compression functionality can be in accordance with the invention employing a back- 
employed with any real time video stream processing of illuminated CCD imager formed on a silicon substrate of 
adequate signal integrity. Thus, as will be recognized by about 10 /xm in thickness and of relatively low resistivity, 
those skilled in the art, many imaging applications can 15 Also shown for reference is the spectral distribution of 
benefit from the dynamic range compression provided by the photon flux for a 2856 degrees Kelvin black body, normal- 
processor, ized to unity at a wavelength of 1000 nm; this black body 

Other digital processing functions beyond adaptive flux is a good approximation to the spectral distribution of 

dynamic range compression are contemplated for use with the night sky illuminance under moonless conditions. As is 

the low-light-level imaging system of the invention. For 20 apparent from the plot, the low-light-level imaging system 

example, imager non-uniformity and defect compensation, of the invention provides a much higher quantum efficiency 

user-specified custom image enhancement, background than the Gen III system at any given wavelength along the 

cancellation, image motion compensation, motion detection spectrum, and provides a peak quantum efficiency of greater 

enhancement, feature extraction, and other image processing than about 90% at a wavelength of about 600 nm. In 

can be undertaken to produce a displayable image in any of 25 dramatic comparison, the Gen III system achieves a peak 

a wide range of formats. This is enabled in the invention by quantum efficiency of no more than about 30% at about 600 

the production of image information in the form of an nm. The plot further illustrates the resulting much broader 

electronic signal, unlike conventional vacuum tube-based overall spectral response of the imaging system of the 

imagers, in which image information is directly converted invention compared with that of the Gen III system, 

from photoelectrons to a phosphor display. 30 Integration of the respective Gen III quantum efficiency 

Turning to the display configuration of the imaging sys- spectrum and that of the imaging system of the invention 

tem of the invention, the display driver 31 (FIG. 1) accepts each against the night sky illumination yields a sensitivity 

the digital image frame sequence from the digital image value about 6111 /iA/lumen for the imaging system of the 

processor and produces corresponding signals appropriate invention and about 1300 ^A/lumen for the Gen III system, 

for the intended display hardware. Suitable display drivers 35 This large sensitivity advantage of the imaging system of the 

include conventional display drivers as provided for use invention is an important factor in obtaining high resolution 

with commercially available displays. As discussed earlier, of low-light-level scenes, as explained in detail below. Use 

the display can consist of any suitable hardware that can of a high-resistivity silicon substrate in the CCD imager 

display a sequence of image frames at a desired frame rate. fabrication process, as described above, instead of a rela- 

For example, monitors, cathode ray tubes, semiconductor 40 tively lower-resistivity silicon substrate, results in a CCD 

displays, and other such displays are acceptable. imager and imaging system that provides even higher quan- 

Finally, the communications link 35 (FIG. 1) of the turn efficiency in the red and near infrared regions of the 

imaging system can consist of any conventional transceiver spectrum than that of the lower-resistivity CCD imager 

system that can support a video signal of a selected band- process; this results in a sensitivity against the night sky 

width over a distance set by a given application for display 45 illumination of as much as about 8076 /iA/lumen, which is 

at the remote location at an intended frame rate. For an even larger sensitivity advantage over the Gen III system, 

example, a conventional wireless transceiver can be The noise characteristics of the low-light-level imaging 

employed to transmit signals from and receive signals at the system of the invention, being very different from that of the 

imaging system of the invention. The communications link Gen III imaging system, results in significant improvement 

can transmit and receive the video imagery in the digital or 50 in overall imaging system noise. The Gen III system, which 

analog domain, but the digital domain is preferred for as explained earlier achieves low-light-level performance by 

limiting noise generation in the signal. means of gain in a micro-channel plate, suffers from the high 

The various components and features of the low-light- noise associated with that gain, although the dark current 
level imaging system of the invention described above work associated with the system is very low and does not con- 
together synergistically to produce a system with superior 55 tribute significantly to the total noise of the system. The 
imaging performance over that of conventional low-light cascading gain mechanism of the micro-channel plate is 
level imagers, and to enable many low-light-level imaging inherently noisy; not only is the gain per cascading photo- 
capabilities that were heretofore unrealizable or impractical. electron "stage" low, but also, the equivalent number of 
In particular, the imaging system of the invention provides stages is itself a statistical variable, resulting in a significant 
substantial advantages over conventional intensifier tube- 60 addition to the noise level. 

based imaging systems, including improved optical In consequence, the Geo III tube has a high noise figure 

sensitivity, improved noise characteristics, improved noise- that is on the order of about 2. Noise figure is here meant as 

limited resolution at low light levels, much greater intra- the ratio of the signal-to-noise level at the input of the 

scene dynamic range, and reduced blooming effects, as well system to the signal-to-noise level at the output of the 

as the capability for electronic image enhancement and for 65 system. This high noise figure imposes a severe noise 

communication of electronic images to locations remote penalty and significantly degrades the low-light-level capa- 

from the imaging system. bility of the Gen III system. In addition, the character of this 
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channel plate noise has been found to be very objectionable; tronic image. With this configuration, both the Gen III 

indeed, it has been found necessary to intentionally reduce system and the imaging system of the invention provided 

the temporal response of the system to avoid motion sick- electronic output images. 

ness in the user This reduced temporal response in turn The two systems were each provided with similar 25 mm 

produces a lag effect that degrades the dynamic resolution of 5 focal -length lenses, set at F/1.4, and the systems were 

the Gen III system. oriented such that they both viewed the same input scene. 

In dramatic contrast, the low-light-level imaging system The input scene was a resolution test pattern characterized 

of the invention provides real time low-light-level imaging by 50% contrast, illuminated at a light level corresponding 

without the use of a gain mechanism — the very high quan- to a starlit scene, i.e., a scene illuminance of about 1.86 

turn efficiency and sensitivity of the CCD imager employed 10 mLUX. The light source was calibrated for the spectral 

in the imaging system enables this. As a result, noise and response of the Gen III image intensifier tube. The light 

other unwanted characteristics of a gain mechanism are source included a filter to block out blue light from the 

entirely eliminated in the imaging system of the invention. scene, to which the CCD imager of the invention, but not the 

Additionally, the low dark current of the CCD imager intensifier tube, would be sensitive; the comparison was thus 

enabled by the dark current perimeter drain, and the further 15 somewhat biased in favor of the Gen III system, 

suppression of dark current using means such as a TEC The images produced by the two systems, both operating 

result in a corresponding CCD imager dark current noise on at 30 frames/second, were displayed side-by-side on the 

the order of only about 5 e"RMS/pixel or less, where RMS same monitor, with the image from the Gen III system 

refers to a root-mean -square measurement. The very low electronically cropped to match the field of display of the 

noise CCD imager output circuits employed in the invention 20 small-format CCD imager. Neither image employed any 

then further limit noise generation as the low-noise signal is digital processing to enhance contrast or compress dynamic 

output. Accordingly, as explained earlier, the CCD imager range. FIG. 21A is a photograph of the monitor image 

produces an output signal that is characterized by a very low produced by the imaging system of the invention, and FIG. 

noise level of less than about 5 e"RMS/pixel at a data rate 2 IB is a photograph of the monitor image produced by the 

per output port of about 2.75 MHz. 25 Gen III imaging system. Comparison of the two images 

The analog signal processing circuitry provided in the shows clearly the superior performance of the imaging 

imaging system at the output of the CCD imager further system of the invention under the starlight conditions of the 

reduces the noise level, as explained previously, by filtering experiment. The dramatically improved low-light-level 

out noise associated with the reset operation of the CCD resolution provided by the invention enables a wide range of 

imager output circuit. Taken together, these various noise 30 low-light- level applications and human activities that are 

suppression and noise reduction mechanisms enable a very only marginally possible, if at all possible, with conven- 

low overall imaging system noise, e.g., on the order of only tional low-light -level imaging systems, 

a few electrons RMS/pixel. This low noise level provides a Referring to FIG. 22, there is shown a plot of the limit of 

superior low- light-level imaging capability that heretofore image resolution as a function of imager illuminance, i.e., 

was unattainable at real time speeds. 35 illuminance on the imager, for the imaging system of the 

In particular, the image resolution capability of the imag- invention and the Gen III system, based on respective 

ing system of the invention is superior to conventional theoretical models for the operational and performance 

imagers at low light levels because image resolution at low characteristics of the systems. The image resolution limit is 

fight levels is a direct function of an imaging system quantified as line-pairs/mm (LP/mm); i.e., the maximum 

signal-to-noise ratio. Specifically, the very low noise levels 40 number of line pairs that can be resolved in a span of one 

of the imaging system of the invention, taken together with millimeter, referred back to the imager, at a given illumi- 

the large CCD imager signal, enabled by its very high nance. The Gen III system characterized in the plot has a 

quantum efficiency, provides a dramatic advantage in signal- noise figure of about 2, a fill factor of about 1, and a 

to-noise ratio over the Gen III system; this results in a sensitivity of about 1300 juA/lumen. The imaging system of 

concomitant advantage in noise-limited resolution at low 45 the invention is characterized by a sensitivity of about 6111 

light levels. //A/lumen and a noise level of about 5e" at a readout rate of 

An operational comparison was performed to compare the about 2 MHz and a controlled temperature of about -20° C. 

low-light-level imaging capability of the imaging system of The imaging system parameters assume 2x2 binning opera- 

the invention with that of the Gen III system. The system tion of the CCD imager below about 20 LP/mm and 4x4 

built in accordance with the invention employed a 128x128 so binning operation of the CCD imager below about 10 

pixel, back-illuminated, frame transfer CCD imager oper- LP/mm. 

ated with four output ports in accordance with the invention, The resolution curves shown in this plot were calculated 

mounted on a TEC within the CCD package, together with using a conventional model, given in, e.g., the RCA Electro- 

an analog signal processor, A/D converter, and image frame Optics Handbook, 1974, p. 196, for low-light-level imaging, 

formater. A second TEC external to the CCD imager pack- 55 the model being modified to include the case of electronic 

age was employed to further reduce dark current charge imaging with finite signal output noise and dark current 

packet generation of the CCD imager. The imaging system noise as well as the inherent photoelectron noise. In order to 

was operated at 30 frames/second, with individual output provide a direct comparison of performance capability of the 

ports each running at about 2.75 MHz, selected to corre- imaging system of the invention with the Gen III system, the 

spond to the data rate required for a practical, large-format 60 plot provides noise-limited resolution as a function of illu- 

640x480 pixel imager operating at 30 frames per second. minance of light impinging the imagers. 

The imaging system of the invention was set up for In the calculation, it is assumed that the resolution pattern 

operation side -by-side with the Gen III system for compar- consists of the standard U.S. Air Force three-bar resolution 

ing the performance of the two systems. The Gen III system test pattern, having a contrast of about 0.3, and an observa- 

was configured with a conventional Gen in, circa-1994 65 tion time of about 0.05 seconds, corresponding to that 

intensifier tube, the output of the tube being optically observation time used in similar calculations by the U.S. 

coupled to a commercial CCD camera to provide an elec- Army Night Vision Laboratories, as given, e.g., by Stefanik, 
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in "Image Intensifier System Resolution Based on Labora- 
tory Measured Parameters," U.S. Army Night Vision 
Laboratory, Ft. Belvoir, Va., August 1994. The calculation 
was made for an imaging system in accordance with the 
invention employing a CCD imager having pixel dimen- 5 
sions of about 12.7 /<mxl2.7 ^m, operating at about 30 
frames per second. 

The range of scene illuminances given in the plot, refer- 
enced to the imager illuminances, as explained below, 
corresponds to the spectrum of conditions typical of low- 10 
light-level imaging applications. Full moon conditions pro- 
vide a scene illuminance of between about 4xl0~ 2 -3xl0~ 1 
LUX, quarter moon conditions provide a scene illuminance 
of between about Ixl0~ 2 -4xl0~ 2 LUX, starlight conditions 
provide a scene illuminance of between about 6xl0~ 4 -2x 15 
10~ 3 LUX, and overcast starlight conditions provide a scene 
illuminance of between about 3xl0~ 3 -3xl0~ 4 LUX. 

The imager illuminance scale of the plot can be refer- 
enced to these natural scenes by specification of a particular 
lens T-number and a particular scene reflectance. For the plot 20 
calculations, a lens T-number of 1.35 was employed; this is 
the T-number typically used in typical night vision systems. 
For convenience, a lens Modulation Transfer Function 
(MTF) of unity was assumed, and the value for the scene 
reflectance was taken as 30%. These choices are exemplary 25 
and somewhat arbitrary; however, it is clear that different 
choices have the effect merely of shifting the scene illumi- 
nance scale with respect to the imager illuminance scale. 
Such a shift in no way affects the relative advantage of the 
imaging system of the invention indicated by the calcula- 30 
tion. 

As shown in the plot, the imaging system of the invention 
provides superior resolution over the Gen III system at any 
imager illuminance condition, referenced to the scene illu- 
mination as discussed above. Under full moon illuminance 35 
conditions and even down to quarter moon illuminance 
conditions, the limit to resolution of the imaging system of 
the invention is set not by the system noise but by the CCD 
imager pixel geometry, at about 40 LP/mm; the Gen III 
system is already noise-limited in resolution at this imager 40 
illumination and achieves only about 24 LP/mm at full moon 
and about 14 LP/mm at quarter moon illuminance. As the 
illuminance is further reduced, the imaging system of the 
invention becomes noise-limited in resolution and achieves 
a resolution of about 20 LP/mm under starlight conditions, 45 
while the Gen III system achieves only half that, i.e., about 
10 LP/mm. In overcast starlight, the imaging system of the 
invention achieves a resolution of about 3.5 LP/mm, while 
the Gen III system achieves only about 1.1 LP/mm. 

Also shown in the plot of FIG. 22 is a straight line 50 
approximation to the bounding envelope of noise-liraited- 
resolution performance of the imaging system of the 
invention, taking into account various possible binning 
modes. This straight line approximation is given by: 

A^-1999Z, a51 , (3) 55 

where is the noise-limited-resolution in LP/mm referred 
back to the imager, and L is the imager illuminance in LUX. 
Comparing the straight line approximation with the calcu- 
lated curves, it is seen that for every illuminance value in the 60 
plotted range, this line defines a minimum noise-limited- 
resolution that is provided by the imaging system of the 
invention and that is not achievable with the conventional 
Gen III system configuration given. 

As indicated in the plot, binning of the charge packets in 65 
the pixels can be employed to increase the noise-limited- 
resolution capability at the lower light levels. In this 
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example with 12.7 i wmxl2.7 pixels, as the light level is 
decreased, and correspondingly, the noise- limited resolution 
decreases, below about 20 LP/mm there is no advantage in 
resolution to be gained from a pixel size smaller than about 
25 /imx25 ^m; and below about 10 LP/mm there is no 
advantage from a pixel size smaller than about 50 /anx50 
pan. In consequence, the CCD imager of the invention can be 
binned in a 2x2 mode below about 20 LP/mm and binned in 
a 4x4 mode below about 10 LP/mm before readout of the 
pixel signal from the CCD imager, with a net improvement 
in noise-limited resolution, as indicated in the plot. 

This resolution comparison clearly illustrates the 
improved low-light-level imaging that is enabled by the 
image system of the invention. It must be emphasized that 
this superior low-light- level resolution performance is pro- 
vided at real time speeds. Thus, the imaging system over- 
comes the limitations of slow-scan and other similar CCD- 
based imaging systems that can resolve images at low levels 
of light but that rely on relatively long CCD integration 
times to produce a viable low- light-level image; real time 
video sequences of a low-light -level image are not possible, 
as a practical matter, with such systems. 

Turning to other advantages of the imaging system of the 
invention, the imaging system is characterized by a very 
large intra-soene dynamic range, as explained above, due to 
the very high quantum efficiency of the CCD imager in 
conjunction with its very low noise characteristic. The 
maximum image signal level for any given scene is set by 
the full-well capacity of the CCD imager of the invention, 
which is about 300,000 e", and the minimum distinguishable 
image signal level is set by the CCD imager readout noise, 
which as stated above, is about 5 e~; as a result, the imaging 
system of the invention is characterized by a dynamic range 
of about 60,000 levels, giving a digital image dynamic range 
representation of about 16 digital bits. 

This is a dramatically increased dynamic range over that 
of conventional tube-based imagers, e.g., the Gen III system, 
which is characterized by a dynamic range of at most about 
200 levels, and which is severely limited by the character- 
istics of the micro-channel plate, as discussed above. The 
imaging system of the invention thus provides an intra -scene 
dynamic range of more than about 200 times greater than 
that of conventional systems. This very large intra-scene 
dynamic range enables imaging of objects both in very dark 
portions of a scene and in bright portions of the same scene 
simultaneously. Recall that the digital image processor pro- 
vided by the invention enables real time compression of the 
very large dynamic range of the image signal to a desired 
subrange, while preserving the object contrast information, 
so that the resulting processed image can be displayed on 
conventional display hardware. 

In addition, the anti-blooming drains in the CCD imager 
of the invention, as described earlier, provide control of an 
image overload up to a factor of about 10 6 , thereby provid- 
ing damage-protection for image overloading such as direct 
imaging of the sun, an intense flare, TV floodlights, or other 
similar overloading conditions. Vacuum tube -based image 
systems suffer from electron redistribution under high illu- 
mination conditions, and are inherently susceptible to dam- 
age through the high light exposure of the photocathode. The 
imaging system of the invention, being based on silicon, is 
inherently rugged and not easily damaged by high light 
overloading. 

The many performance advantages outlined in this 
discussion, in combination with the portability, flexibility, 
and customization provided by the imaging system of the 
invention, result in a low-light-level imaging system that 
heretofore has been unattainable. 
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In an alternative embodiment of the low-light-level imag- 
ing system of the invention, two or more CCD imagers, in 
conjunction with real time computer processing, are config- 
ured in a low-light -level imaging system that simultaneously 
images multiple fields of view at different resolutions and 
presents them to the user as a single blended image. This 
configuration is motivated by the nonuniform sampling 
characteristics of retinal photoreceptors; indeed, the eye 
samples a scene at much higher resolution in the central part 
of the field of view, and sampling resolution degrades into 
the peripheral field of view. The central field of view is thus 
used for tasks requiring high visual acuity, like reading and 
shape recognition, whereas the peripheral field of view is 
important for detecting fast moving objects and capturing 
the user's attention, but is not useful for shape recognition. 

This suggests a strategy whereby a low-light-level imag- 
ing system can be endowed with a. central field of view, at 
high resolution, that is presented at unity magnification on 
an output display, as well as a wide peripheral field of view 
that is presented at reduced resolution on the display, with 
the two fields of view blended together. The central imaged 
field would subtend an angle typical of conventional low- 
light -level imagers, i.e., approximately 40 degrees or more. 
The peripheral field of view can extend quite a bit further, 
e.g., to 100 degrees, but would appear minified on the 
display; yet it would provide greater visual context than the 
central field alone, and a moving object in the periphery 
could capture a user's attention, who could then turn the 
low- light-level imaging system towards the object of interest 
so as to image that object with the high resolution part of the 
field of view. 

Referring to FIG. 23, this is achieved in the invention, in 
one example, to provide a wide field of view-multiresolutioo 
imaging system 400 that includes two CCD imagers each 
conforming to the performance standards described earlier; 
the imagers can be of a planar or curved profile, but a 
preferable combination is described below. The example 
system shown is portable and provides an eyepiece 406 for 
a user to focus his gaze 408 directly on a display 410 in the 
system in alignmeot with one of the CCD imagers 402. 

In the system operation, the axes of the two imagers are 
aligned on a scene to be imaged. Incident light 412 is split 
by a beam splitter 414. In a preferable configuration, one of 
the imagers 404 is mounted on a planar substrate and uses 
a longer focal length lens 416 to image a central field of 
view, typically 40 degrees or more; while the second imager 
402 is mounted on a curved substrate and uses a short focal 
length lens 418 to image a wide field of view, possibly 100 
degrees. Both imagers can have similar or equal numbers of 
pixels, but the field of view per pixel is very different for the 
two imagers. 

The beam splitter can embody, e.g., a typical 50/50 beam 
splitter, but this is not the most efficient Instead, it is 
preferred that, as shown, the beam splitter 414 is apertured 
and angled at 45 degrees to the lines of sight to the two 
imagers, which are preferably angled at 90 degrees to one 
another, with a central area of high reflectivity so as to image 
that portion of the field of view onto the narrow field high 
resolution imager, and a clear outer area so as to transmit the 
peripheral field of view to the wide field low resolution 
imager. 

The two CCD imagers produce analog image signals 420, 
422, that in one example configuration are input to a portable 
image processing computer 424 for digitization and digital 
image processing; the digitization and digital image pro- 
cessing can alternatively be provided by separate compo- 
nents. The resulting two digital images are manipulated by 
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the computer 424 so as to blend the two together, with the 
narrow field high resolution image in the center of the 
composite image, and the wide field low resolution image 
surrounding the central image area so as to form a peripheral 

5 part of the composite image. The resulting composite image 
426 is output to the display 410, on which it is displayed to 
the user with unity magnification at the central high reso- 
lution area, and with less than unity magnification for the 
peripheral low resolution area. With this operation, the 

10 display must subtend a field of view, with respect to the user 
viewing it, that exceeds the central narrow field but that may 
be less than the peripheral wide field. 

The wide field peripheral image can be minified 
uniformly, or it can be compressed in spatial extent 

15 nonuniformly, say with increasing minification towards the 
extreme edges of the wide field of view. The central and 
peripheral fields will be smoothly blended together in the 
preferred implementation. Preferably, this combining of 
central and peripheral fields of view occurs at about real time 

20 speeds with a latency time that does not exceed one frame 
time. Any one of a wide range of conventional image 
blending algorithms can be employed in the invention to 
achieve the desired field of view blending in real time. 
In one embodiment of this system, two CCD imagers, 

25 each of 512x512 pixels, are aligned so as to image a scene 
with a central field of 40 degrees and a peripheral field of 
100 degrees. The two fields produced by the imagers are 
combined as described above, with increasing minification 
of the peripheral field towards the outer edge of the field. 

30 The composite image is presented on a display supporting 
768x768 pixels, subtending a user's field of view of 60 
degrees, say. In this case, the central 512x512 pixels of the 
display present the central field of 40 degrees, i.e., 20 
degrees around the axis, at unity magnification, with reso- 

35 lution of 0.078 degrees/pixel, and the outer "ring" of 128 
pixels presents the 30 degree peripheral annular field which 
extends from 20 degrees to 50 degrees from the axis, with 
average resolution of 0.234 degrees/pixel, or one-third the 
average resolution of the central field. 

40 As will be recognized by those skilled in the art, the 
analog, digital, timing, and other processing and control 
components described earlier can be applied to the wide 
field of view-multiresolution imager to produce a very high 
performance low-light-level imaging system. 

45 Beyond imaging multiple fields of view, the imaging 
system of the invention provides the ability to image mul- 
tiple spectral ranges, over a registered field of view, to 
produce a color image of the scene to user. For example, two 
CCD imagers can be employed in the imaging system, with 

50 each CCD imager imaging a distinct spectral range, e.g.,. 
visible and near-infrared ranges, respectively. The two 
images produced by the CCD imagers are here fused by the 
digital image processor to produce a full color display 
image. A suitable example of this fusion functionality is 

55 described by Waxman et al. in U.S. patent application Ser. 
No. 08/332,696, filed Nov. 1, 1994, the entirety of which is 
hereby incorporated by reference. 

As will be recognized by those skilled in the art, three 
CCD imagers could alternatively be employed to produce a 

60 full color image. In this case, each of the three CCD imagers 
is designated as either the red, green, or blue color channel 
of the display. Preferably, each of the three channels is 
controlled to independently accommodate scaling, as is 
conventionally performed for so-called white balancing of 

65 an image. In the invention, selected spectral ranges are 
directed to the multiple CCD imagers by suitable means 
such as dichroic beam splitters. 
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In an alternative scenario, the imaging system employs ues from the center surround-shunt processor to a 

only one CCD imager that itself images multiple spectral preselected output dynamic range represented by the 

ranges. This is accomplished with any of several configu- number of digital bits D; and 

rations provided by the invention. In a first example a remap processor connected to receive the enhanced and 

configuration, the imaging pixel array of the CCD imager is 5 normalized pixel values from the center surround-shunt 

divided into clusters of pixels, with each pixel in a given processor and connected to the remapping function 

cluster designated for a given spectral range. This can be processor to apply a remapping function to the 

accomplished, as is common in the art, using a correspond- enhanced and normalized pixel values for rescaling 

ing spectral filter layered over each pixel. In an example of each enhanced and normalized pixel value to the output 

this configuration, pixels designated for the red spectral 10 dynamic range and to produce at the frame rate, R, and 

range are layered with a filter through which only light of the with a latency time of no more than about 1/R, a 

red spectral range substantially passes; similarly for green- sequence of output image frames of remapped pixel 

and blue-designated pixels, appropriate filters can be layered values. 

over corresponding pixels. 2. The image processing system of claim 1 further com- 

The spectral range of light that is not passed through a 15 pri smg a display connected to receive the output digital 

given spectral filter is lost to the imager, resulting in lower image f rame sequence for displaying the sequence at the 

performance of the imager. An imager configuration pro- frame rate, R. 

vided by the invention that circumvents this limitation 3. The image processing system of claim 1 wherein the 

employs diffractive elements above each cluster of pixels to number of digital bits, D, that represent the output dynamic 

direct light of the appropriate spectral range towards corre- 20 range) ^ less than the number of digital bits, B, that represent 

sponding individual pixels in the cluster. Diffractive ele- the dynamic range of the input digital image frames, 

ments such as those commonly used in the field of binary whereby the image processing system compresses the 

optics can be employed here to achieve the spectral range dynamic range of the input digital image frames, 

directionality. 4. The image processing system of claim 2 wherein the 

The foregoing description has been set forth to illustrate 25 number of digital bits, D, that represent the output dynamic 

the many features and advantages of the invention and is not range, is selected to correspond to displayable dynamic 

intended to be limiting. Since modifications of the described ran g e 0 f the display on which the output image frames are 

embodiments incorporating the spirit and substance of the to be displayed. 

invention may occur to persons skilled in the art, the scope 5. The image processing system of claim 3 wherein the 

of the invention should be limited solely with reference to 30 number of digital bits, B, representing the input digital frame 

the appended claims and equivalents thereof. dynamic range, is greater than 8. 

We claim: 6. The image processing system of claim 5 wherein the 

1. An image processing system for adaptively processing number of digital bits, D, representing the compressed 

a sequence of input digital image frames each represented by dynamic range of remapped pixel values is no larger than 8. 

an array of digital pixel values corresponding to a scene 3S 7. The image processing system of claim 6 wherein the 

image and having a dynamic range represented by a number number of digital bits, B, representing the input digital frame 

of digital bits, B, to produce at a frame rate, R, a sequence dynamic range, is 12, and wherein the number of digital bits, 

of digital output image frames each represented by an array d, representing the compressed dynamic range of remapped 

of digital output pixel values having an output dynamic values, is 8. 

range represented by a number of output digital bits, D, the ^ The image processing system of claim 1 wherein the 

system comprising: frame rate, R, is at least about 25 frames per second, 

a center-surround-shunt processor connected to receive 9. The image processing system of claim 1 wherein the 

the sequence of input digital image frames at the frame remapping function constructed by the remapping function 

rate, R, and for each frame to evaluate each pixel value processor comprises a look-up table arranged with look-up 

based on neighboring pixel values in that frame's pixel 45 table addresses, each address having a corresponding pixel 

value array to produce a corresponding enhanced pixel value, and correspondingly wherein each enhanced pixel 

value that results in enhanced local spatial contrast in value is evaluated by the remap processor as a look-up table 

the scene image, the center-surround-shunt processor address and the look-up table pixel value corresponding to 

adaptively normalizing each resulting frame of that address is output as the corresponding remapped output 

enhanced pixel values to a prespecified pixel value 50 pixel value. 

range; 10. The image processing system of claim 1 wherein the 

a statistics processor connected to receive the sequence of neighboring pixel values on which the center-surround shunt 

input digital image frames to determine for each input processor bases evaluation of an input frame pixel value 

frame a measure of mean scene image brightness, and comprises a Gaussian weighted local neighborhood of pixel 

connected to receive the enhanced frame pixel values to 55 values in the corresponding input frame, 

determine for each enhanced frame a measure of mean 11. The image processing system of claim 1 wherein the 

enhanced image brightness and a measure of enhanced normalized pixel value range to which the center-surround- 

pixel value standard deviation across the enhanced shunt processor normalizes each frame of enhanced digital 

frame; pixel values is -1 to +1. 

a remapping function processor connected to receive for 60 12. The image processing system of claim 1 wherein the 

each input frame the measure of mean enhanced frame center-surround-shunt processor is limited in its evaluation 

image brightness and the measure of enhanced pixel pixel values in an input frame to produce only corresponding 

value standard deviation across the corresponding enhanced pixel values that result in enhanced positive local 

enhanced frame and in response to construct a pixel spatial contrast in the scene image, 

value remapping function based on the received 65 13. The image processing system of claim 1 wherein: 

measures, the remapping function constituting a rule each frame in the input digital frame sequence is desig- 

for remapping the enhanced and normalized pixel val- nated F,-, where i is the frame number in the sequence; 
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the measure of mean scene image brightness determined 
by the statistics processor for frame F, is employed by 
the center-surround shunt processor to produce 
enhanced pixel values for frame F lV1 ; and 

the measure of mean enhanced frame image brightness 5 
and the measure of enhanced pixel value standard 
deviation for frame F t is employed by the remapping 
function processor to construct a pixel value remapping 
function for frame F i+2 . 

14. The image processing system of claim 1 wherein the 10 
input digital frame sequence corresponds to a visible scene 
image characterized by an arbitrary scene illuminance. 

15. The image processing system of claim 1 wherein the 
input digital frame sequence corresponds to an X-ray scene 
image. 15 

16. The image processing system of claim 1 wherein the 
input digital frame sequence corresponds to an infrared 
scene image. 
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17. The image processing system of claim 1 wherein the 
input digital frame sequence corresponds to a magnetic 
resonance scene image. 

18. The image processing system of claim 1 wherein the 
input digital frame sequence corresponds to a radar scene 
image. 

19. The image processing system of claim 1 wherein the 
input digital frame sequence corresponds to an acoustic 
scene image. 

20. The image processing system of claim 1 wherein the 
input digital frame sequence corresponds to a thermal scene 
image. 

21. The image processing system of claim 1 wherein the 
input digital frame sequence corresponds to an ultraviolet 
scene image. 

22. The image processing system of claim 1 wherein the 
input digital frame sequence corresponds to a millimeter 
wave scene image. 

* * * * * 
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